03 Apr 10 jQuery UI Autocomplete Widget & Caching
I upgraded to the awesome new 1.4 version of jQuery recently in preparation for deploying another beta version of my site. Unfortunately the jquery-autocomplete plugin by Jörn Zaefferer doesn’t work with this new version. However, the equally awesome jQuery user interface library, jQuery UI, now has a new autocomplete widget that is based upon Jörn’s design. The API isn’t quite the same, but after some reading you’ll find that most of the functionality is still there. However the new UI widget does not support caching requests, leaving that to the user to implement. This kind of surprised me, as I would think that would be a commonly requested feature. But perhaps the UI folks felt that everyone would have different caching requirements. There is one example on the UI site of how to cache, but curiously it only caches the last request, which doesn’t seem all that useful to me.
In any event, after reading the documentation, it wasn’t hard to implement caching for my purposes. The following code simply caches the last 16 requests. Once the cache is full, it is flushed completely and ready to be filled again. Simple, crude, but hopefully somewhat effective. I’m not a Javascript expert, but this will work for me. I hope someone else finds it useful and can use it as a starting point for their use-case.
$(function() {
var cache = {};
var cacheSize = 0;
$("#id_of_your_text_box").autocomplete({
delay: 400,
minLength: 2,
source: function(request, response) {
if (cache[request.term]) {
response(cache[request.term]);
return;
}
$.ajax({
url: "/your/url/goes/here/",
type: "GET",
data: {
q: request.term,
limit: 10
},
dataType: "json",
success: function(data, textStatus) {
if (cacheSize >= 16) {
cache = {};
cacheSize = 0;
}
cache[request.term] = data;
++cacheSize;
response(data);
},
error: function(xhr, textStatus, ex) {
alert('Oops, an error occurred. ' + xhr.statusText + ' - ' +
xhr.responseText);
}
});
}
});
});
The key to making this to work is to provide your own source function to the autocomplete widget. The widget will call this function when it needs data. Inside the function we simply check to see if request.term is already in our cache. If it is, then we simply return the cached result via the response callback function. If not, we have to make an ajax call to the server to retrieve the data. When the ajax call completes, we store the result in our cache and then return it to the widget.
Again, this code simply stores the last 16 requests. When we run out of room we simply empty the cache and start over. A more sophisticated algorithm could use a different strategy like removing only the first item put in the cache or even the “least recently used” cache item. Break out your college computer science textbook on caching strategies and go nuts here.
The other feature this code doesn’t do is provide matching capabilities. For example, if the user has typed “foo” in the input box, this code will not return results for “bar foo”. If you need this, I suggest looking at the caching example on the UI site, or even at Jörn Zaefferer’s source code.
Tags: autocomplete, javascript, jquery, jquery-ui, plugins
08 Mar 09 Event Calendar: Submitting Events
I’m continuing on in my experiment to leverage Google calendar for my site’s event calendar. I have built a form that allows users to submit new events subject to admin approval. I modeled the form after the GUI used by Google calendar. I also decided to used jQuery UI’s datepicker. jQuery UI just released version 1.7, which is compatible with jQuery 1.3. After reading this blog entry, I’ve decided to try letting Google host jQuery for me. Google is also hosting the standard jQuery UI themes CSS files, so this seems awfully convenient.
Check out the screenshot of the event submit form:
The datepicker is very useful, and will add a lot to the site, I think. My existing PHP application is using a much less elegant form. So far this form only handles non-repeating events. I will add support for those later.
I then turned my attention to the backend: how to synchronize my event models with Google calendar? It turns out there is already a Django application designed for this purpose: django-cal. It is designed in a very generic way and seems to be very well done. I decided I didn’t need its full functionality, but I did reference the source code when designing my back end. So a big thank-you to the creators of django-cal!
At this point I have enough functionality to add events and get them on the Google calendar. I have coded, but not tested, update and delete functionality. I am using the batch facility of the gdata API. In a typical work cycle, I need to approve anywhere from 1 to 7 or so events at once. This isn’t a lot of events, but I think doing it in a batch mode will speed a synchronization cycle up noticeably.
I store a “status” field in each event that keeps track of the event status: new, new and approved, edited, edited and approved, deleted, deleted and approved, and “on calendar”. When a user adds an event it becomes “new”. An admin reviews the event, and marks it “new and approved”. After synchronization, the status changes to “on calendar”. Likewise, my plan is to let users edit and request deletes for any events that they submit. My custom admin synchronization view will the perform the operations on all events with the “… and approved” status in a batch mode.
The custom admin view I wrote about in the last entry is also working well. I overrode the base django.contrib.admin template, so it looks well-integrated with the rest of the admin part of the site. I also figured out how to use the breadcrumbs for admin navigation, and noticed the base template will display a variable called “messages” with the nice green checkmark, if present. Likewise, the admin CSS has a class called “errorlist” for displaying errors. So it is quite easy to make a pretty professional looking custom admin screen by studying the base template and admin CSS files. I will post a screenshot of my custom admin view after I get some more events through the work cycle in a future blog post.
The Python gdata API isn’t as clumsy as I first thought it was. I was able to wrap it in a Calendar class to make it easier to use. If you need to update or delete an event in batch mode, it seems like you have to retrieve the event from Google first. I struggled against this initially, since I was storing away the “edit link” for each event I create. I was hoping to use the edit link to avoid another trip to Google for an update. But after searching the Google Calendar Data API group archives, it seems that is indeed recommended to retrieve any event prior to an update or delete, at least in batch mode.
This seems to be working out well so far. After I code and test the other operations I will blog with more details.
14 Feb 09 Member Map Part 2
I’ve gotten the Member Map application ported to my Django powered site now. This was pretty straightforward and a lot of fun because of the Javascript aspect. Let me address the points I made in part 1 of this post, below.
- Leveraging jQuery. I did make use of jQuery for the client side Javascript. This really did simplify things. In the old version of the code, I had a 282 line Javascript file. The new version weighs in at 161 lines, which is 57% the size of the earlier version.
- Use of JSON encoder library. It turns out that Django includes a version of the simplejson library already. This is used in Django’s serialization feature. I probably could have used this feature, but I didn’t need to serialize an entire model, and wanted to rename some of the fields. So I simply did a “import django.utils.simplejson as json” and went to town. My server side code was now much simpler than the previous version where I encoded by hand.
- I did continue to use the markItUp! editor to let users edit their message to go along with their location. I learned that the size of the editor is easily adjusted through the use of CSS. So I was able to override the default size of the editor by including my own CSS style sheet below the default sheet.
- Reducing complexity. Here is where it got interesting. I decided to add a “json” field to the MapEntry model. This field is populated whenever a model object is saved by providing my own save() method. Inside my save() I use Django’s template render_to_string() to render the HTML message that appears in the pop-up balloon on the map. This consists of the user’s avatar followed by a message that is marked up in Markdown. I then build a JSON representation of the MapEntry using Django’s simplejson. This representation consists of the following fields: name, location, latitude, longitude, and the HTML message. By pre-saving all of this information, it makes it much more efficient to retrieve 200+ users’ information when the map is first loaded. In the view function, I make use of the values_list() queryset method to retrieve all the json fields. I don’t need the ORM to convert each MapEntry to an actual Python object.
Another thing I did differently for the port was when a user updates or adds their position on the map, I don’t reload the entire map. I just adjust their information to reduce bandwidth with an AJAX post. I’m not sure why I didn’t do this before, perhaps because it was to tedious to do without jQuery. It was just easier to reload the whole thing.
This was all well and cool, doing all this pre-computation, but what happens when a user changes her avatar? Her entry on the map will likely have a broken graphic. No problem. Here I took advantage of Django’s signals. I attached a signal handler to listen for changes to the UserProfile model. Whenever the UserProfile is saved, my signal handler runs and re-saves the corresponding MapEntry to regenerate the JSON. Very slick.
And finally, another thing that I learned about Django was that simple_tags can have defaulted arguments. I added an argument to my avatar tag so that I could apply CSS to the generated HTML img tag if needed. In the Member Map pop-up balloon, it looked nicer if the avatar was floated to the left.
So all and all, again, I’m very happy with how easy Django makes writing an application like this. Using jQuery was also a big productivity booster.
Here are some screenshots. You can also see the new Blueprints CSS in action as well.
Member Map 1
Member Map 2
06 Feb 09 Member Map Part 1
I started working on the Member Map application last weekend. This will be a port of the PHP-Nuke module I wrote, which you can find here. In a nutshell, this application displays a Google Map on your site, and allows site members to place markers representing their location on the map. If you click on a marker, a balloon pops up which displays the user’s avatar, a link to their profile, and a short message that they have typed previously. The Nuke version of this application is a big hit on the current site, and last time I checked, we had over 220 members on the map.
The existing application consists of the PHP-Nuke code, written in PHP, on the server side, and a fair amount of Javascript on the client side. All interaction with Google Maps is done through the Javascript code.
For the port to Django, I want to do a couple of things different.
- Leverage jQuery in the client-side Javascript. The existing version was written before I was aware of Javascript libraries, and it is all “vanilla” Javascript. By using jQuery, the amount of code I have to write will be much smaller. And by taking advantage of jQuery, it should be more portable and browser robust.
- On the server side, the existing application did not use any JSON encoder library. I simply constructed the JSON by hand. I can’t remember why I did this. I think it was partly because I didn’t know any better, and partly because the PHP version I had on the original hosting server may not have had it. I will explore what Python and/or Django options exist.
- I’ll leverage the markItUp! editor that I’ve been using for comments to let the users enter Markdown comments. I’ve used this in the last couple of apps I’ve written and it will be easy to implement. The only concern with doing this is the user may be tempted to write a lot more than in the older app, and it may make for large “balloons”. We’ll see.
- I want to make the new application less database and computationally intense than the older version. I don’t have any problems running the current version with 200 users, but I think I could do it a bit smarter this time around. This means I am going to try to save marked up and JSON encoded strings in the database rather than compute them on every page load. I am also wary of instantiating 200+ map entry model objects with the Django ORM (albeit in a loop) just to generate the JSON. Luckily Django lets you use raw SQL when needed. But in this case I think a simple use of the values() method will really be useful here.
So, those are my going in goals. I’ve already started working on this, and I’ve learned a lot so far. In the next post or two I will detail the progress and what problems I ran into.
24 Dec 08 Anatomy of a Shoutbox; Part 6 – Conclusion: Delete In Place
Welcome to part 6, the final post on the development of the new shoutbox. In case you missed the earlier parts, here they are: part 1, part 2, part 3, part 4, part 5. Here we are going to discuss the final feature: deleting a shout in place off the shout history page.
The idea here is to add a delete link on the shouts that the user created as she is viewing the shout history. We will use the jQuery library to dynamically add click handlers to these links. The click handler functions will make an AJAX post to the server and request that a shout be deleted. The server view function will verify that the user sending the request actually created the shout in question. If so, the shout will be deleted from the database and then a response sent back to the client. This response will be then be used by our Javascript to hide the deleted shout. Thus we will have deleted a shout without refreshing the entire page.
The first thing I did was modify the shout history template to output a link underneath a shout if the current user authored that shout. This template (shoutbox/view.html) was presented in part 5. I’ll just show the new code inside the for loop that adds these links:
{% ifequal user shout.user %}
| <a href="#" class="shout-del" id="shout-del-{{ shout.id }}">Delete</a>
{% endifequal %}
The important thing to note is our links don’t go anywhere, they have the CSS class “shout-del”, and we give them an id of “shout-del-xx” where xx is the id of the shout from the database.
I then modified the shoutbox_app.js document ready function to look for all links that have the class “shout-del” and I attach a click function to them. The click function asks the user via the Javascript confirm() function if they really want to delete the shout. If they confirm, I then use javascript regular expression magic to pull out the shout’s primary key from the <a> tag’s id, and send a POST request to the server to delete that shout. The relevant snippet is shown below.
$('.shout-del').click(function () {
if (confirm('Really delete this shout?')) {
id = this.id;
if (id.match(/shout-del-(\d+)/)) {
$.post('/shout/delete/', { id : RegExp.$1 }, function(id) {
id = '#shout-del-' + id
$(id).parents('tr').hide();
$('div.shoutbox-history table tr:visible:even').removeClass('odd');
$('div.shoutbox-history table tr:visible:odd').addClass('odd');
}, 'text');
}
}
return false;
});
We are posting to the URL /shout/delete, and passing along the shout ID in the id parameter. We also supply a callback function that gets called when the delete happens successfully. More on that later. Finally the click function returns false so that the default action of clicking on a link is not executed.
We now turn to the server, and examine the view function that handles the URL /shout/delete. You’ll notice similar code as presented in part 5′s edit view function. This time I have discovered Django’s classes that derive from HttpResponse, and I use them in the error cases.
def delete(request):
"""This view deletes a shout. It is called by AJAX from the shoutbox history view."""
if request.user.is_authenticated():
id = request.POST.get('id', None)
if id is None:
return HttpResponseBadRequest()
try:
shout = Shout.objects.get(pk=id)
except Shout.DoesNotExist:
return HttpResponseBadRequest()
if request.user != shout.user:
return HttpResponseForbidden()
shout.delete()
return HttpResponse(id)
return HttpResponseForbidden()
We check to see if the user is authenticated before proceeding. We then retrieve the shout ID from the POST QueryDict. From the ID we retrieve the actual shout from the database using the Django ORM. And finally, if and only if the user making the request is the same user that created the shout, we delete it. If all of that went well, we send back a positive HTTP response that includes the ID of the shout that was deleted.
Now refer back to our Javascript, posted above. In the callback function we receive the ID as text from the server. We then convert the numeric ID to the CSS ID and use jQuery to select the parent <tr> of our <a> tag, and hide it. Wow! That was a very compact way to do all of that work.
The remaining logic takes care of the table presentation. I used CSS to style every other row in the shout history table with a class called “odd”. Since we just hid one row, that scheme got messed up. Luckily, the compact notation of jQuery again comes to the rescue and we can quickly restyle all odd visible rows with our class “odd”, taking care to remove the style from the new even rows first.
And there we have a shoutbox written in Django and Javascript for the new version of my site. Not bad for my first attempt, and it certainly was much easier than I imagined. All in all, there is a lot less Python code than the existing PHP code, and we even get neat-o AJAX’y features that the old one didn’t have. So I think we will go with this for the first pass of the site. Once I get more comfortable with jQuery I may decide to redo the scrolling itself with it. We could also look into posting shouts to the database without a complete page refresh. There is always something to go back and refactor as you learn more about your tools and best practices. I hope someone found this quick series of posts useful.
20 Dec 08 Anatomy of a Shoutbox; Part 5 – Edit In Place
In this post I’ll show how I got an edit-in-place feature working with the shoutbox. In case you missed the earlier parts, here they are: part 1, part 2, part 3, part 4.
The first thing I did was create a “shout history” view. This allows you to view the shouts in a full page context. In my views.py file I created this:
def view(request, page=1):
"""This view allows one to view the shoutbox history."""
paginator = DiggPaginator(Shout.objects.all(), SHOUTS_PER_PAGE, body=5, tail=3, margin=3, padding=2)
try:
the_page = paginator.page(int(page))
except InvalidPage:
raise Http404
return render_to_response('shoutbox/view.html', {
'page': the_page,
},
context_instance = RequestContext(request))
Here I am using the DiggPaginator, which is a very nice paginator that gives you “Digg-style” pagination. I then pass the page object to the template which does most of the interesting work.
In order to get the edit-in-place funcitonality to work, I am leveraging the Jeditable jQuery plugin. In order to use this plugin, in my shoutbox/view.html template I include <script> tags to bring in both the jQuery javascript library and the Jeditable plugin code. I then loop over the shouts for the page and display them along with the avatar and links to the user that made the shout. You’ll also see me using the smilify fiilter I discussed in part 4.
