Upgrading Trac on Windows Gotcha - Part 2
I have previously reported on some problems I had when upgrading our Trac install at work. Today I attempted another upgrade to Subversion 1.7.4 and Trac 0.12.3 on Windows. I upgraded to Python 2.7.2 along the way. I ran into another problem with the Python bindings to Subversion which took a while to figure out. I had no problems upgrading Subversion, but Trac could not see our repository. The symptoms were that the Trac "timeline" and "browse source" features were missing.
Following the Trac troubleshooting advice, I opened an interactive Python session and tried this:
E:\Trac_Data>python
Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> from svn import client
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\svn\client.py", line 26, in <module>
from libsvn.client import *
File "C:\Python27\lib\site-packages\libsvn\client.py", line 25, in <module>
_client = swig_import_helper()
File "C:\Python27\lib\site-packages\libsvn\client.py", line 21, in swig_import
_helper
_mod = imp.load_module('_client', fp, pathname, description)
ImportError: DLL load failed: The operating system cannot run %1
After some head scratching and googling I finally found the problem. I had used the Windows .msi installer, graciously provided by Alagazam, aka David Darj, to install Subversion. This placed the Subversion binaries and DLL's in C:\Program Files\Subversion\bin. I then unzipped the Python 2.7 bindings to the C:\Python27\Lib\site-packages folder. The bindings depend on the DLL's in the Subversion\bin folder. But unfortunately for me, there were already two older versions of the DLL's, libeay32.dll and ssleay32.dll on my path. So when the bindings went looking for those two DLL's, instead of finding them in Subversion\bin, it found the older versions somewhere else.
To fix this, you can either rearrange your path, or copy those two DLL's to your Python27\Lib\site-packages\libsvn folder. In the future, I am going to just copy all the DLL's from Subversion\bin to the libsvn folder.
I examined the pre-built Subversion packages from Bitnami and CollabNet. They had packaged all of the Subversion DLL's with the Python bindings together in the same directory, so this seems reasonable. Later, on the Subversion users' mailing list, Alagazam gave the nod to this approach.
A big thank you to Alagazam for the help and for the Windows binaries. And of course thanks to the Apache, Subversion, Trac, & Python teams for making great tools.
Upgrading Trac on Windows Gotchas
At work, we are outfitted with Windows servers. Despite this obstacle, I managed to install Trac and Subversion a few years ago. During a break in the action, we decided to update Subversion (SVN) and Trac. Since we are on Windows, this means we have to rely on the kindness of strangers for Subversion binaries. I ran into a couple of gotchas I'd like to document here to help anyone else who runs into these.
I managed to get Subversion and Trac up and running without any real problems. However when Trac needed to access SVN to display changesets or a timeline, for example, I got an error:
tracerror: unsupported version control system "svn" no module named _fs
After some googling, I finally found that this issue is documented on the Trac wiki, but it was kind of hard to find. To fix this problem, you have to rename the Python SVN binding's DLLs to *.pyd. Specifically, change the libsvn/*.dll files to libsvn/*.pyd, but don't change the name of libsvn_swig_py-1.dll. I'd really like to hear an explanation of why one needs to do this. Why doesn't the Python-Windows build process do this for you?
The second problem I ran into dealt with mod_wsgi on Windows. Originally, a few years ago, I setup Trac to run under mod_python. mod_python has long been out of favor, so I decided to cut over to mod_wsgi. On my Linux boxes, I always run mod_wsgi in daemon mode. When I tried to configure this on Windows, Apache complained about an unknown directive WSGIDaemonProcess. It turns out that this mode is not supported on Windows. You'll have to use the embedded mode on Windows.
Contributing to open source - a success story and advice for newbies
Recently, my team at work found a bug in Subversion, I submitted a patch, and it was accepted! This was very exciting for me so I thought I would share this story in the hopes of inspiring others to contribute to open source projects. It may not be as hard as you might think!
The Bug
We use Subversion at work for revision control. My colleague and I were trying to merge a branch back to trunk when we ran into some strange behavior. We make use of Subversion properties, which allow you to attach arbitrary metadata to files and directories. Our project has to deliver our source code and documentation to the customer in a required directory format (can you guess who our customer is?). However not all files need to be sent to the customer. To solve this problem we use a simple "yes/no" delivery property on each file to control whether it is delivered or not. Before making a delivery, a script is run that prunes out the files that have the delivery flag set to "no".
When our merge was completed, many files were marked with having merge conflicts on the delivery property. Looking through the logs it was discovered that after we had made our branch, someone had changed the delivery property on some files to "yes" on the trunk. Someone else had also changed the delivery property independently to "yes" on the branch. When we attempted to merge the branch back to trunk, we were getting merge conflicts, even though we were trying to change the delivery property value to "yes" on both the trunk and branch. Why was this a conflict? This didn't seem quite right.
I signed up for the Subversion user's mailing list and made a short post summarizing our issue. I later learned that it is proper etiquette to attach a bash script that can demonstrate the problem. Despite this, a Subversion developer took interest in my post and created a test script in an attempt to reproduce our issue. At first it looked like he could not reproduce the problem. However another helpful user pointed out a bug in his script. Once this was fixed, the developer declared our problem a genuine bug and created a ticket for it in the issue tracker.
The Patch
Impressed by all of this, I thanked him for his efforts and tentatively asked if I could help. The developer told me which file and function he thought the problem might be in. I downloaded the Subversion source and began looking at the code. I was fairly impressed with the code quality, so I decided I would try to create a patch for the bug over the weekend. We really wanted that bug fixed, and I was genuinely curious to see if I would be able to figure something out. It would be an interesting challenge and a test of my novice open source skills.
When the weekend came I began a more thorough examination of the Subversion website. The Subversion team has done a great job in providing documentation on their development process. This includes a contributing guide and patch submittal process. I also discovered they had recently added a makefile that downloaded the Subversion source code and the source for all of Subversion's dependencies. The makefile then builds everything with debug turned on. Wow! It took me a few tries to get this working, but the problems were because I did not have all the development tools installed on my Ubuntu box. Once this was sorted, everything went smoothly, and in a matter of minutes I had a Subversion executable I could run under the gdb debugger. Nice!
I studied the code for about an hour, peeking and poking at a few things in the debugger. I used the script the developer wrote to recreate the problem. I wasn't quite sure what I was doing, as I was brand new to this code base. But the code was clearly written and commented well. My goal was to get a patch that was in the 80-100% complete range. I wanted to do enough work that a core developer would be able to see what I was doing and either commit it outright or easily fill in the parts that I missed. After a while I thought I had a solution and generated a patch. I sent it to the Subversion developer's mailing list as per the contributing guide.
The Wait
Next I began probably the worst part for a contributor. I had to wait and see if I got any feedback. On some open source projects a patch may languish for months. It all depends on the number of developers and how busy they are. My chances didn't look good as the developers were in the initial stages of getting a beta version of 1.7 out the door. It was also not clear to me who "owned" the issue tracker. On some projects, the issue tracker is wide open to the community. Was I supposed to update the ticket? I wasn't quite sure, and the contributing guide was silent on this issue. I eventually concluded I was not; it looked like only committers were using the tracker. Patches were being discussed on the mailing list instead of in the tracker. This is a bit different than some projects I am familiar with.
I didn't have to wait long. After a few days, the original developer who confirmed my bug took interest again. He looked at my patch, and thought I had missed something. He suggested a change and asked for my opinion. I looked at the code again; it seemed like a good change and I told him I agreed. I also warned him I was brand new to the code, and to take my opinion with a grain a salt. After running my change against the tests, he then committed my patch! One small victory for open source!
Final Thoughts
So what went right here? I have to hand it to the Subversion team. They have been in business a long time, and they have excellent documentation for people who want to contribute. The makefile they created that sets up a complete development environment most definitely tipped the scale for me and enabled me to create my patch. Without that I'm not sure I would have had the time or patience to get all that unfamiliar source code built. The Subversion team has really worked hard at lowering the barrier for new contributors.
My advice to people who want to contribute to open source but aren't quite sure how to go about doing it:
- Spend some time reading the documentation. This includes any FAQ's and contributor guides (if any).
- Monitor the user and developer mailing lists to get a feel for how the community operates. Each project has different customs and traditions.
- You may also wish to hang out on the project's IRC channel for the same reason.
- When writing on the mailing lists, be extremely concise and polite. You don't want to waste anyone's time, and you don't want to be seen as someone who thinks they are entitled to a fix. Just remember you are the new guy. You can't just barge in and make demands.
- Ask how you can help. Nothing makes a developer happier when someone asks how they can help. Remember, most of the people in the community are volunteers.
- Open source can sometimes be "noisy". There will be people who won't quite understand your issue and may hurriedly suggest an incorrect solution or give incomplete advice. Study their responses and be polite. You may also wish to resist the temptation to reply right away. This is especially hard when you are new and you don't know who the "real" developers are. However you should assume everyone is trying to help.
- Finally, be patient. Again, most folks are volunteers. They have real jobs, families and lives. The project may also be preoccupied with other tasks, like getting a beta out the door. Now may not be a good time for a brand new feature, or your bug may not be considered a show stopper to the majority of the community.
A big thank-you to Stefan Sperling from the Subversion team who shepherded my bug report and patch through their process.
I hope this story encourages you to contribute to open source software!
