Continuing along with the Google calendar integration, I built views to allow users to edit and delete the events they have previously submitted. These actions are simply requests; the admin still has to approve them. In the PHP version, these updates did not require admin approval. Here, they do, because the admin must supply the Google password to synchronize the events on the Google calendar. So the events simply get marked as “edit request” or “delete request”, which the admin must approve by toggling the status to “edit approved” or “delete approved”. The admin then visits the custom Google sync view, supplies the password, and the events are synchronized with Google via the gdata API. This seems to be working out well, although I’m not entirely happy with the workflow in the admin area to accomplish all this. Right now I have to visit the model change list, approve changes, then go to the custom Google sync view. We’ll see how this works out in practice.
I was very pleasantly surpised at how easy it was to add a link to my custom Google sync view in the admin area. I decided I wanted this link visible on all model views in the admin area (there is only one model right now: the event). I simply had to create this path in one of my template directories: admin/gcalendar/change_list.html (where gcalendar is the name of my application). Django will look to see if I have provided a change_list.html, and will use it in preference to the default one. And I didn’t even have to copy the whole template, I only needed to use template inheritance to override the block that provides the “object tools”. My change_list.html looks like this:
{% extends "admin/change_list.html" %}
{% block object-tools %}
{% if has_add_permission %}
<ul class="object-tools"><li><a href="add/{% if is_popup %}?_popup=1{% endif %}" class="addlink">Add {{ name }}</a></li>
<li><a href="google_sync/">Google Sync</a></li>
</ul>
{% endif %}
{% endblock %}
I just added the <li> and link to my “Google Sync” page. Here is a screen shot of my calendar event change list that shows this link in action. Check the upper right, next to the “Add” link. It was nice to see it was automatically styled to fit in with the rest of the admin.
Clicking on the link leads one to the custom admin view. Here, the list of approved events appears. The admin types in the Google password, and the view synchronizes (adds, updates, deletes) the events with Google. It might be a minor pain to enter your Google password every time, but I’d rather not hard-code or store the admin’s Google password in the database.
I think using Google calendar is going to be cool. I like the fact that people can add the events to their own calendars, invite other people, and set email or SMS text message reminders for these events. I also like that other people can embed the site calendar on their own websites or blogs to spread the word.
With that in place, all I have left to do is to support repeating events. And actually, as it stands right now, I can also handle multi-day events. This covers 99% of the events on SurfGuitar101.com already. There are only two weekly gigs on the calendar right now. I have worked on this off and on for 3 or 4 weeks now, and have replicated most of the PHP version’s functionality by leveraging Google and Django. It took me about 3 or 4 months before I had this much working when I initially developed the PHP-Nuke version.
Repeating events might be difficult, because the gdata API sort of punts on them. In order to specify a repeating event, I have to format a fairly complex string according to some iCalendar RFC. Yikes. I will now poke around and see if there are some Python libraries available that I could use for this, or perhaps it isn’t as bad as I think it is and I can do it by hand. Or perhaps I will leave repeating events for the future since they aren’t used that much. I am itching to get what I have created so far deployed so I can start beta-testing and getting feedback from users.
I’m continuing on in my experiment to leverage Google calendar for my site’s event calendar. I have built a form that allows users to submit new events subject to admin approval. I modeled the form after the GUI used by Google calendar. I also decided to used jQuery UI’s datepicker. jQuery UI just released version 1.7, which is compatible with jQuery 1.3. After reading this blog entry, I’ve decided to try letting Google host jQuery for me. Google is also hosting the standard jQuery UI themes CSS files, so this seems awfully convenient.
Check out the screenshot of the event submit form:
The datepicker is very useful, and will add a lot to the site, I think. My existing PHP application is using a much less elegant form. So far this form only handles non-repeating events. I will add support for those later.
I then turned my attention to the backend: how to synchronize my event models with Google calendar? It turns out there is already a Django application designed for this purpose: django-cal. It is designed in a very generic way and seems to be very well done. I decided I didn’t need its full functionality, but I did reference the source code when designing my back end. So a big thank-you to the creators of django-cal!
At this point I have enough functionality to add events and get them on the Google calendar. I have coded, but not tested, update and delete functionality. I am using the batch facility of the gdata API. In a typical work cycle, I need to approve anywhere from 1 to 7 or so events at once. This isn’t a lot of events, but I think doing it in a batch mode will speed a synchronization cycle up noticeably.
I store a “status” field in each event that keeps track of the event status: new, new and approved, edited, edited and approved, deleted, deleted and approved, and “on calendar”. When a user adds an event it becomes “new”. An admin reviews the event, and marks it “new and approved”. After synchronization, the status changes to “on calendar”. Likewise, my plan is to let users edit and request deletes for any events that they submit. My custom admin synchronization view will the perform the operations on all events with the “… and approved” status in a batch mode.
The custom admin view I wrote about in the last entry is also working well. I overrode the base django.contrib.admin template, so it looks well-integrated with the rest of the admin part of the site. I also figured out how to use the breadcrumbs for admin navigation, and noticed the base template will display a variable called “messages” with the nice green checkmark, if present. Likewise, the admin CSS has a class called “errorlist” for displaying errors. So it is quite easy to make a pretty professional looking custom admin screen by studying the base template and admin CSS files. I will post a screenshot of my custom admin view after I get some more events through the work cycle in a future blog post.
The Python gdata API isn’t as clumsy as I first thought it was. I was able to wrap it in a Calendar class to make it easier to use. If you need to update or delete an event in batch mode, it seems like you have to retrieve the event from Google first. I struggled against this initially, since I was storing away the “edit link” for each event I create. I was hoping to use the edit link to avoid another trip to Google for an update. But after searching the Google Calendar Data API group archives, it seems that is indeed recommended to retrieve any event prior to an update or delete, at least in batch mode.
This seems to be working out well so far. After I code and test the other operations I will blog with more details.
I ran into something that took a few hours to sort out, so I thought I would write it up here in case it would help some other poor dope out.
My goal is to create a custom admin view. I noticed that the Django docs had been updated recently, documenting a new member function called get_urls() on the ModelAdmin class. The docs are kind of sketchy right now, but I gathered that this was the new way to add custom admin views for a model. However, before you can use this new feature, you have to cut over to the new scheme of hooking the contrib.admin site into your project. This is what took me a few hours to figure out, as I hadn’t noticed this change in the docs.
This is how I had my URLConf in my Django 1.0 project:
urlpatterns = patterns('',
(r'^admin/(.*)', admin.site.root),
# ...
After updating my working copy of Django trunk, this is what my URLConf needed to look like to use the new admin get_urls() scheme:
urlpatterns = patterns('',
(r'^admin/', include(admin.site.urls)),
# ...
All the changes are on line 2. Notice, there are two changes you have to make:
After finally figuring these two things out, I am happy to report that the get_urls() function on my ModelAdmin class worked as I expected. For example, if I use the regular expression ‘^my_view/$’ for my custom admin view, the view will be located at /admin/appname/modelname/my_view. This isn’t quite made crystal clear in the Django docs.
Now I am poised to add a custom admin view on my Google Calendar application. Stay tuned for more on that.
I haven’t had a lot of time to explore implementing an event calendar with Google Calendar, but I did try a few things out. First of all, it is pretty easy to embed a Google Calendar into your website:
As I mentioned in the last post, I downloaded the Google Python gdata client library. I wrote a small Python script based on the provided example code. I was able to add events to my calendar from this Python script:
import gdata.calendar
import gdata.calendar.service
import gdata.service
import atom
import atom.service
import time
cal_client = gdata.calendar.service.CalendarService()
cal_client.email = 'xxx@gmail.com'
cal_client.password = 'xxx'
cal_client.source = 'Google-Calendar_Python_Sample-1.0'
cal_client.ProgrammaticLogin()
title = 'My Test Event from Python'
content = '''This is the content of the event in some fashion.
<a href="http://google.com">My Link</a>
http://surfguitar101.com.
<b>Bold</b> text.
<p>Here is a paragraph.</p>
<p>And another</p>
<ul><li>Item 1</li><li>Item 2</li></ul>
<img src="http://surfguitar101.com/modules/Forums/images/smiles/icon_cool.gif" alt="Smiley" />
'''
where = 'An undisclosed location.'
#cal_path = '/calendar/feeds/default/private/full'
cal_path = '/calendar/feeds/xxx@group.calendar.google.com/private/full'
event = gdata.calendar.CalendarEventEntry()
event.title = atom.Title(text=title)
event.content = atom.Content(text=content)
event.where.append(gdata.calendar.Where(value_string=where))
# Use current time for the start_time and have the event last 1 hour
start_time = time.strftime('%Y-%m-%dT%H:%M:%S.000Z', time.gmtime())
end_time = time.strftime('%Y-%m-%dT%H:%M:%S.000Z', time.gmtime(time.time() + 3600))
event.when.append(gdata.calendar.When(start_time=start_time, end_time=end_time))
new_event = cal_client.InsertEvent(event, cal_path)
As you can see, I was testing to see if I could insert HTML into the event, and if it would display correctly on the Google Calendar. According to my tests, it does appear you can insert simple HTML. The only thing that did not work was <img> tags. Unfortunately, if you log into your Google account, and then go to your Google Calendar and try to edit this event using the Google GUI, all HTML will be stripped. This will make it difficult to edit an existing event and will probably steer me towards keeping the user added event around in the database for future editing until that event has expired.
I have to say I am not really impressed with the Python API. It seems rather “un-Pythonic”. Why do I have to construct atom objects and assign them to the event title and content fields? The API should do this for me. Why do I have to format my own date strings? The API should accept Python datetime objects and do that for me. Pretty disappointing coming from Google.
I am starting to see a way forward here. I think I need to do the following:
Initially I will support only non-repeating events. I’ll go back and try to add support for repeating events in “phase 2″ once I get the above functionality working.
I was looking through the Django docs and came across a new feature in trunk that allows you to extend the set of URLs supported by a model in the admin section. This will allow me to create a custom admin view to insert the events. I think this is just a new way of doing something that was already supported, and is described in the online Django book. I’ll have to update my working copy of Django trunk and try this out.
I wrote a pretty complex event calendar for my PHP-Nuke site. You can find it here on sourceforge. It allows users to submit events for admin approval. Once approved they appear on the calendar. The application does all the calendar logic, including the drawing of the calendar. Repeating events were added, and they are pretty hairy as well. There are some pretty massively complicated SQL queries in that code, not to mention some crazy PHP logic to figure out if repeating events belong on a particular calendar view. It is also possible to make exceptions to repeating events.
I’ve been thinking about how to port this to my new Django powered site. One day I stumbled across the Google Calendar API, and I’m thinking about leveraging this. I will let Google handle all the crazy calendar logic, as they are no doubt better at it than I am. I will still let users submit events to the admin, but this time I will use the Google API to add events to a Google Calendar. Google can do the heavy lifting for me when it comes to displaying events and handling repeating events. Am I cheating by doing this? Ha. Well there are some advantages to this idea for my end users:
So I’ve decided to explore this approach. If it doesn’t look like it will pan out, I will fall back to a more direct port of my old PHP code.
This approach isn’t without its own challenges. I can see right now I am going to have to think carefully about how to provide authentication to the Django code to allow it to add events to one of my Google calendars. I obviously don’t want to hard code my Google login and password in the Django application.
I think I will attack this problem in phases:
I’ve already downloaded the Python version of the gdata library and have started reading the API docs. It looks interesting. Once I get familiar with this library, it looks like I could use it for other cool things like interfacing with YouTube and Google Docs.
I’ve gotten the Member Map application ported to my Django powered site now. This was pretty straightforward and a lot of fun because of the Javascript aspect. Let me address the points I made in part 1 of this post, below.
Another thing I did differently for the port was when a user updates or adds their position on the map, I don’t reload the entire map. I just adjust their information to reduce bandwidth with an AJAX post. I’m not sure why I didn’t do this before, perhaps because it was to tedious to do without jQuery. It was just easier to reload the whole thing.
This was all well and cool, doing all this pre-computation, but what happens when a user changes her avatar? Her entry on the map will likely have a broken graphic. No problem. Here I took advantage of Django’s signals. I attached a signal handler to listen for changes to the UserProfile model. Whenever the UserProfile is saved, my signal handler runs and re-saves the corresponding MapEntry to regenerate the JSON. Very slick.
And finally, another thing that I learned about Django was that simple_tags can have defaulted arguments. I added an argument to my avatar tag so that I could apply CSS to the generated HTML img tag if needed. In the Member Map pop-up balloon, it looked nicer if the avatar was floated to the left.
So all and all, again, I’m very happy with how easy Django makes writing an application like this. Using jQuery was also a big productivity booster.
Here are some screenshots. You can also see the new Blueprints CSS in action as well.
I started working on the Member Map application last weekend. This will be a port of the PHP-Nuke module I wrote, which you can find here. In a nutshell, this application displays a Google Map on your site, and allows site members to place markers representing their location on the map. If you click on a marker, a balloon pops up which displays the user’s avatar, a link to their profile, and a short message that they have typed previously. The Nuke version of this application is a big hit on the current site, and last time I checked, we had over 220 members on the map.
The existing application consists of the PHP-Nuke code, written in PHP, on the server side, and a fair amount of Javascript on the client side. All interaction with Google Maps is done through the Javascript code.
For the port to Django, I want to do a couple of things different.
So, those are my going in goals. I’ve already started working on this, and I’ve learned a lot so far. In the next post or two I will detail the progress and what problems I ran into.
I’ve been meaning to start paying attention to the SQL queries that Django is generating for me. I’d like to know if there are any dumb things I am doing. And down the road I am going to start employing low-level caching, and seeing what queries I am doing will help me decide what things to cache. I finally broke down and figured out how you can display the SQL queries used during an HTTP request on the rendered web page. Django makes this quite easy and it is fully documented.
If your TEMPLATE_CONTEXT_PROCESSORS in your settings.py file contains “django.core.context_processors.debug”, and DEBUG is set to True, you have access to a variable in your templates called sql_queries. This is an ordered list of {'sql': ..., 'time': ...} dictionaries that represent every SQL query and how long each one took during the processing of the request.
It was quite easy to add the following to the very bottom of my site’s base template:
{% if debug %}
<div id="debug">
<ol>
{% for s in sql_queries %}
<li>{{ s.sql }} : <b>({{ s.time }})</b></li>
{% endfor %}
</ol>
</div>
{% endif %}
Now I can keep an eye on what’s happening at the database level. I’ve already added a few key “select_related” clauses to some of my Django ORM queries that have saved 10-30% on the number of trips to the database per page. Cool!
I sat down and pounded out quite a bit of code last weekend, which is kind of unusual. I want to summarize what happened and leave some notes for myself.
On the comments front, I worked commenting into both the news and polls applications. This was astonishingly easy as I had already done the work once for the photo of the day. Django’s content types framework is really nice. Having the ability to attach comments to arbitrary models is just brilliant.
It occurred to me that I should take advantage of Django forms media, and then I wouldn’t have to repeat the several lines of <link> and <script> tags I have to add to every page that uses the markItUp! editor. I added a “get_comment_form for” template tag, and then I thought I could just do a “{{ form.media }}” at the top of the template and then use the same form variable at the bottom of the template. But this did not work. I believe it has something to do with using the form variable in separate blocks in my templates that extend my base template. I posted a question about this on the django-user’s list, but did not receive a response. Oh well, not a huge deal, but I must investigate why this didn’t work sometime.
The other weird thing that happened was my Firefox browser started lurching to a halt when the shoutbox was being displayed with my new CSS scheme. I swear it didn’t do this before with the older CSS. So I got mad and disabled the Javascript scrolling for now, leaving it with a scroll bar (which actually doesn’t look that bad to me). I think this is only a problem on the Linux Firefox, I have not seen this on Windows. I have noticed that some sites give Linux Firefox some trouble if they have Javascript. It seems to eat an inordinate amount of CPU, making things like paging up and down a chore. I started looking at some jQuery-based tutorials for an alternate means for scrolling but didn’t find anything that jumped out at me right away. Again, I’ll put that on the to-do list.
On the CSS front, I came across the Blueprint CSS framework. This looks interesting for a design dolt like myself. It may help me with the overall site structure.
And finally last night I started working on the downloads application. This is my last “easy” application. After that I have to work on the calendar, the member map, Paypal, and the forums. The downloads application will be very similar to the Web links application that I did early on. It will contain many of the same types of views and sorting options. I am going to try and do it a bit smarter this time as I have learned a lot since then. I want to add an AJAX style star rating system and comments. I may then go back and refactor Web links with what I have learned. Ideally I should be looking for ways to share the code. If Django had a class-based way of handling views I think I could see a way forward. I could also finally break down and investigate generic views.