Here is a quick update on a previous post I made about a newline-to-break extension for Python-Markdown. I'm very happy to report that the extension will now be shipping with Python-Markdown! Thanks to developer Waylan Limberg for including it!

When I launched a new version of my website, I decided the new forums would use Markdown instead of BBCode for the markup. This decision was mainly a personal one for aesthetic reasons. I felt that Markdown was more natural to write compared to the clunky square brackets of BBCode.

My new site is coded in Python using the Django framework. For a Markdown implementation I chose Python-Markdown.

My mainly non-technical users seemed largely ambivalent to the change from BBCode to Markdown. This was probably because I gave them a nice Javascript editor (MarkItUp!) which inserted the correct markup for them.

However, shortly after launch, one particular feature of Markdown really riled up some users: the default line break behavior. In strict Markdown, to create a new paragraph, you must insert a blank line between paragraphs. Hard returns (newlines) are simply ignored, just like they are in HTML. You can, however, force a break by ending a line with two blank spaces. This isn't very intuitive, unlike the rest of Markdown.

Now I agree the default behavior is useful if you are creating an online document, like a blog post. However, non-technical users really didn't understand this behavior at all in the context of a forum post. For example, many of my users post radio-show playlists, formatted with one song per line. When such a playlist was pasted into a forum post, Markdown made it all one giant run-together paragraph. This did not please my users. Arguably, they should have used a Markdown list. But it became clear teaching people the new syntax wasn't going to work, especially when it used to work just fine in BBCode and they had created their playlists in the same way for several years.

It turns out I am not alone in my observations (or on the receiving end of user wrath). Other, much larger sites, like StackOverflow and GitHub, have altered their Markdown parsers to treat newlines as hard breaks. How can this be done with Python-Markdown?

It turns out this is really easy. Python-Markdown was designed with user customization in mind by offering an extension facility. The extension documentation is good, and you can find extension writing help on the friendly mailing list.

Here is a simple extension for Python-Markdown that turns newlines into HTML <br /> tags.

"""
A python-markdown extension to treat newlines as hard breaks; like
StackOverflow and GitHub flavored Markdown do.

"""
import markdown


BR_RE = r'\n'

class Nl2BrExtension(markdown.Extension):

    def extendMarkdown(self, md, md_globals):
        br_tag = markdown.inlinepatterns.SubstituteTagPattern(BR_RE, 'br')
        md.inlinePatterns.add('nl', br_tag, '_end')


def makeExtension(configs=None):
    return Nl2BrExtension(configs)

I saved this code in a file called mdx_nl2br.py and put it on my PYTHONPATH. You can then use it in a Django template like this:

{{ value|markdown:"nl2br" }}

To use the extension in Python code, something like this should do the trick:

import markdown
md = markdown.Markdown(safe_mode=True, extensions=['nl2br'])
converted_text = md.convert(text)

Update (June 21, 2011): This extension is now being distributed with Python-Markdown! See issue 13 on github for the details. Thanks to Waylan Limberg for the help in creating the extension and for including it with Python-Markdown.