13 Dec 09 Python Subprocess Popen and Windows
I was writing a Python script to automate some Subversion-related tasks at work last week on Windows. The Python code was calling the Subversion command-line client applications using the Python subprocess module. I don’t know why exactly, but I had passed in the parameter shell=True to the subprocess.Popen function. I probably did this because I saw a lot of examples on the web, and you know, monkey-see, monkey-do. In hindsight, these examples were probably for Unix. It turned out that this caused a lot of problems. The Subversion clients started complaining when handed filenames that contained “special” characters like ampersands. And when they got to the files that had both ampersands and spaces, all kinds of crazy errors were produced. I was pulling my hair out, trying to escape the special characters and/or use quotes around the filename, but I could not come up with a combination that would work for all the different filenames the script encountered. Finally, as I stared at the code, I somehow got the idea to try switching that shell=True to shell=False. Suddenly, everything worked perfectly. I undid all the escaping and quoting and everything worked as expected.
So I’m not sure what shell=True does on Windows, and it certainly isn’t documented. I do not recommend using it unless you know what is happening in the implementation. Interestingly, shell=False is the default. I should have left it that way. 
Tags: python, subprocess, svn, windows
06 Apr 09 Infrastructure: Trac & Subversion
I’ve been wanting to get some kind of issue tracker up and running for some time now. Trac seems like a great choice. We’ve used it where I work, and the Django project uses it. I even managed to install it on Windows at work. Still, I was kind of dreading trying to get it working on the dedicated server I rent. I finally gathered the strength and tackled this problem this weekend, and it went far easier than I imagined.
Subversion
First of all, I decided I might as well upgrade my Subversion (SVN) server while I am at it. I see that Subversion 1.6 is out now. However, reading the fine print, I noticed that they seemed to have changed their Python bindings in 1.6, and I wasn’t sure if Trac is compatible with this. So without doing any further research I decided to just run the last stable version before that, 1.5.6.
My dedicated server is running Fedora Core 6, which isn’t maintained anymore, so there is no way to my knowledge of getting a binary package for these recent builds of SVN. I need to build from source. I had done this once before, and I even took detailed notes (which I had forgot about). Building from source is fairly easy, but there is one gotcha on the AMD64 server I run, you need to invoke the configure script with an –enable-shared switch. Luckily I wrote this down from the first time I did this. Getting the required dependencies for the source build isn’t too hard. The Subversion folks helpfully package some of the less readily available dependencies, so it is just a matter of grabbing them and untarring them on top of the unpacked source tarball.
Since I wanted to integrate Trac with my Subversion repository, I needed to ensure I built the Python Subversion bindings. I used Yum, the package manager that comes with Fedora, to make sure I had SWIG installed before I ran configure to build SVN. Then it is a simple matter of building the Python SWIG bindings after Subversion proper is built. This is explained very well in the Subversion documentation.
This seemed to go well, although I had a minor heart attack when Apache crashed the first time I tried to restart it with the new SVN in place. Another restart and it was fine. Hmmm. In short order I had upgraded my existing repository and things seemed to be working fine.
A New Subdomain
I then created a new subdomain to host my issue tracker. I rely on the Plesk control panel to do this lifting for me. It came installed with the server, and I rely on it heavily to configure Apache, the mail server, etc. I’m not a hard core server admin, so this is a big help. Although I can see the day when the training wheels can come off as I become more familiar with Linux and these tools. I can sort of see what Plesk is doing by examining the config files it creates and it doesn’t appear to be rocket science. Still, it is a big time saver for me.
Trac
To get Trac installed requires getting all the dependencies in place first. In most cases, I was was able to use Yum to get most of the dependencies in binary form from the Fedora repository. Despite the fact that Fedora Core 6 is pretty old, the version numbers of the dependencies in the repository were still compatible with the newest version of Trac. The one notable exception was the template engine Trac uses, Genshi. In this case a simple “easy_install Genshi” did the trick. Nice.
I might have been able to easy_install Trac, but the docs say that this only works for Python 2.5 and 2.6. I’m still running 2.4 on the dedicated server. Upgrading my OS is definitely on the long term to-do list, but I must take baby-steps for now. But it was a simple matter of grabbing the Trac tarball, untarring it, and doing the usual “python setup.py install”. It went flawlessly.
Now luckily I had setup Trac at work before, so I already knew what to do. I ran the command-line Trac admin tool to create a project and tied it to my new Subversion repository. Trac comes with a development server, and I ran that after configuring the project. I could then point my browser at my server and see my new Trac project for the first time! Things are cooking at this point.
Mod_WSGI
Of course I can’t use the development server for real work. So the next step was to get Apache to serve my Trac project. I once again chose mod_wsgi as the deployment method, after just recently converting The Madeira site from mod_python to mod_wsgi. The mod_wsgi documentation is excellent, and a wiki page covers integrating Trac and mod_wsgi in great detail. After studying the docs for a short while I had the magic Apache configuration down. I restarted Apache, and once again I was amazed that things were working on the first try. I had been pretty lucky so far. (In fact the most trouble I had that day was trying to change the logo on the Trac site!)
At Last…
I was now ready to configure my Trac project and get my new Subversion repository loaded. I had an existing Subversion repository that I was doing all my work in. However I had checked in some settings files that contained database password information. Shortly after realizing this I just locked the whole repository down. Since then, I have learned the Django settings.py and local_settings.py trick, and have placed the sensitive information in the local_settings.py file (which is not controlled in SVN). Now I can have a public read-only repository again.
So here it is, ready for beta testing: http://code.surfguitar101.com. Now there isn’t anything stopping me; I have to do the real work of deploying a beta version of SG101 2.0 for testing and feedback.
