msgbartop
by Brian Neal
msgbarbottom

22 Jan 10 Testing Your First Django App

Dougal Matthews wrote a great blog post entitled “Testing Your First Django App“. This is something that I have been meaning to do for a long time now, but didn’t know how to get started. Much to my surprise (I guess I shouldn’t be), Python supports the xUnit style of testing via the standard library package unittest. Since we are now using CxxTest at work, this is quite familiar to me now. Dougal’s blog entry shows some nice ways of testing Django web applications without using a server by mocking up requests and examining responses from your view functions. Very cool!

Tags: , , ,

31 Dec 09 Python and SSH

I was fooling around with my webcam, and I got to wondering if I could write a small Python script to upload a snapshot to my webserver periodically. Since I was planning on uploading a new snapshot every ten minutes or so, I’d rather use SFTP to avoid sending my login and password unencrypted so frequently. I had already used FTP in Python before in a backup script, but I didn’t know if the Python standard library had a solution for SSH. After a bit of Googling, I discovered the 3rd party Python module Paramiko (the name is a combination of the Esperanto words for “paranoid” and “friend” — I love that), which allows you to do all kinds of SSH2 operations, including SFTP. Paramiko is extremely well documented (thank-you!) and I had no problems at all using it to accomplish my task. I also found this great introductory article by Python core developer Jesse Noller that was helpful in getting up to speed on it quickly.

Since I was using my webcam from Windows XP, I decided to write my script under the new 1.7.1 version of Cygwin that came out recently. You’ll need to get both the Paramiko package and the python-crypto package.

Happy New Year everyone!

Tags: , , , , ,

13 Dec 09 Python Subprocess Popen and Windows

I was writing a Python script to automate some Subversion-related tasks at work last week on Windows. The Python code was calling the Subversion command-line client applications using the Python subprocess module. I don’t know why exactly, but I had passed in the parameter shell=True to the subprocess.Popen function. I probably did this because I saw a lot of examples on the web, and you know, monkey-see, monkey-do. In hindsight, these examples were probably for Unix. It turned out that this caused a lot of problems. The Subversion clients started complaining when handed filenames that contained “special” characters like ampersands. And when they got to the files that had both ampersands and spaces, all kinds of crazy errors were produced. I was pulling my hair out, trying to escape the special characters and/or use quotes around the filename, but I could not come up with a combination that would work for all the different filenames the script encountered. Finally, as I stared at the code, I somehow got the idea to try switching that shell=True to shell=False.  Suddenly, everything worked perfectly. I undid all the escaping and quoting and everything worked as expected.

So I’m not sure what shell=True does on Windows, and it certainly isn’t documented. I do not recommend using it unless you know what is happening  in the implementation. Interestingly, shell=False is the default. I should have left it that way. :)

Tags: , , ,

30 Nov 09 Using the Python Shell in Vi Mode

The Python shell is an invaluable tool for trying things out and experimenting with the language. Wouldn’t it be great if you could use the Python shell in Vi mode? Much to my surprise, it turns out you can.

Python leverages the GNU Readline Library in its shell code. The Readline library is used in a number of open source projects, notably the BASH shell. The Readline library looks for a configuration file at startup in a number of places. On Unix-like operating systems, the configuration file can be found at ~/.inputrc and falls back to /etc/inputrc.

One line is all you need in your ~/.inputrc file:

set editing-mode vi

Unfortunately I don’t know how to make this work under Windows.

Tags: , ,

04 Sep 09 Denyhosts: SSH Brute Force Protection

I’m always very uneasy when I look through my web server and system logs. It seems dozens of people (or more!) a month try to log into my server, attempt an SQL injection attack on one or more of my hosted websites, and probe for weaknesses. It really makes me lose some faith in humanity.

I was tipped of to the Denyhosts script via a tweet from @dozba. This is nothing more than a Python script that analyzes your ssh log file, looking for suspicious activity. It monitors the number of failed SSH login attempts over time, and once a configurable threshold is reached, it adds the IP address of the remote host to your hosts.deny file, thus blocking the brute-force break-in attempt. The script is very configurable. It can have different thresholds for login attempts to existent or non-existent user accounts, it can purge blocked entries, and it will email you when it blocks a host.

Installation and configuration on Ubuntu is a breeze, as it is available as a package. Use your favorite package manager and install it. Ubuntu automatically configures it to run in daemon mode, so you won’t need to mess with cron. Configure it by editing the file /etc/denyhosts.conf. Restart the daemon, and then you are done. In a nutshell:

$ sudo apt-get install denyhosts
$ sudo vi /etc/denyhosts.conf
$ sudo /etc/init.d/denyhosts restart

Two days after I installed it, I had already blocked 5 break-in attempts. Thanks to the author Phil Schwartz and the Ubuntu package maintainers for this very useful utility!

Tags: , , ,

24 Jul 09 Call of Duty Random Map Rotation Generator Script

Here is a Python script I wrote to randomly generate a map rotation for our Call of Duty: World at War server. It outputs the “set sv_mapRotation” line that you should copy & paste into your server’s config file. Our gaming clan mainly plays TDM, but we got a second server to try out the other objective-based game types. Not knowing which map combinations were the best, I simply wrote a script to generate a random rotation of the non-TDM and non-DM gametypes. Unfortunately, I found out later that there is a 1024 character line length limit in the config file (boo!). So I later modified the script to observe this unfortunate limit. This means you can’t have very large map rotations.

Here are some notes about the script.

  • The script should work with any Call of Duty game from 1 to W@W. You just have to edit the list of maps and use the appropriate game types. Just be advised that the game type names aren’t consistent across titles. For example, COD:MW uses “WAR” for team deathmatch, while the older games, and COD:W@W use the more familiar “TDM”. Not all game types are supported in all titles in the series, and some titles have game types that don’t exist in other games. E.g. Only COD1 has retrieval (“RE”) and behind enemy lines (“BEL”).
  • Put the desired maps you want between the triple quoted lines 9 and 14, and separate them by spaces.
  • The gametypes  tuple on line 16 contains the game types supported. Edit this to contain the game types you want.
"""
map_rotate.py 

Generate map rotation for COD:W@W.
25 June 2009 - Brian Neal
"""
import random

maps = """\
mp_airfield mp_asylum mp_castle mp_shrine mp_courtyard mp_dome mp_downfall
mp_hangar mp_kneedeep mp_makin mp_nachtfeuer mp_outskirts mp_roundhouse
mp_seelow mp_subway mp_suburban mp_makin_day
mp_docks mp_kwai mp_stalingrad
""".split()

gametypes = ('dom', 'koth', 'sab', 'sd', 'ctf', 'twar')

combos = []
for gametype in gametypes:
   for map in maps:
      combos.append((gametype, map))

random.shuffle(combos)

# There is a 1024 character limit for lines from the config file...damn.

s = 'set sv_mapRotation "'
for game in combos:
   ss = 'gametype %s map %s ' % (game[0], game[1])
   if len(s) + len(ss) > 1024:
      break
   s += ss

s = s.rstrip()
s += '"'

print s

Remember that the script is random, so you may get the same map back-to-back, and/or some maps may be missing due to the stupid 1024 character limit. I run it a few times, saving the result into a file, until I get one that looks good. If you are ambitious, you could modify it a bit to prevent these issues, although the code for doing something like that starts to get very complicated.

I hope some people find this useful!

Tags: , , ,