Lamson 1.0pre2, HTML Email, Standalone

Lamson 1.0pre2 features two features that might signal the end of the beginning or
the beginning of the end, depending on your perspective: HTML Email and Lamson Standalone.
HTML Email support comes from a new module
lamson.html that gives a nice template
method to send out HTML to victims…uh…customers. Lamson Standalone will be a way
to run Lamson as your customized email server instead of another server like Postfix.

Both of these features will be done for Lamson 1.0, but are currently just getting started
as of 1.0pre2. Once they’re done that will be the last two requested features people
had before 1.0.

Getting This Release

As usual, you can get this release from /releases or if you are too lazy to
read that page then do this:

$ sudo easy_install -U lamson

In addition, if you want play with the new features you’ll need to install these (optional)
Python libraries:

$ sudo easy_install -U pydns
$ sudo easy_install -U beautifulsoup
$ sudo easy_install -U markdown2
$ sudo easy_install -U clevercss

I’ll decide if I should make these mandatory or optional with the next release based on
people’s feedback.

Lamson Standalone

Quite a few people have asked for a way to install Lamson as their primary email server.
Usually they want it for just their own email on a machine just for them where installing
something like Postfix is overkill or just too hard. This is entirely possible
with Lamson, but it’s kind of not the right use for it since simple delivery of mail is
much better handled and implemented by an older server like Postfix.

Rather than resist giving people a way to setup Lamson as their primary mail server, I’ve
decided to start making it possible for them. By the next release (1.0pre3) you’ll be able
to run Lamson as your only email server for small installations handling mail for one person.

I’ll also include a screen cast showing people how to use the standalone functionality
to build a Personal Mail Management Server to filter and control their mail without any
other mail server. I’m going to treat this as a first project for most people wanting to get
started using Lamson for their email.

As of 1.0pre2, the functionality that’s available is code in the lamson.server.Relay
that will use PyDNS to query up the MX host for
recipient addresses. It’s fairly primitive right now, but you use it by creating
a Relay with the host explicitly None:

relay = server.Relay(host=None, port=25, debug=1)

Once you do this the Relay will lookup the hosts rather than trying to send through a relay
host.

Don’t go crazy with this yet, since it has to be tested with various kind of nasty email
addressing out there, and it needs to have a way to generate a bounce when it has an error.

HTML Email

Well, I broke down and implemented my idea for making HTML Email easy as hell to generate.
In the past I didn’t want to include simple HTML support because, well, HTML Email is
annoying as hell and I didn’t want to deal with the support headaches. I knew once I threw
HTML generation into Lamson I’d have an army of marketing people using Lamson poorly to
generate their marketing materials. I also think that HTML formatting in email doesn’t work
as a customer development strategy.

Yet, every time I tell someone about Lamson, the very first, second, third, and 300th thing they
ask is if it does HTML Email. Over and over and over this was the most important feature, above
spam blocking, filtering, building applications, intelligent state management, or anything else
that Lamson supports.

Well, if the people want HTML email generation, then the people will get it. I introduce
you to lamson.html which
makes it trivial to produce HTML in your email and doing it in a nice clean way using
CleverCSS and Jinja2
templates.

Let’s start with the simplest little example that will send out a disgusting html template:


import sys
from lamson import html, server
from config import testing

relay = server.Relay(host=None, port=25, debug=1)
hs = html.HtmlMail(“style.css”, “html_test.html”)

title = “Test Message HTML”

msg = hs.respond(locals(), “content.markdown”, From=sys.argv1, To=sys.argv2, Subject=“Test %(title)s”)

relay.deliver(msg)

You could run this right out of the Lamson source tree like this:

export PYTHONPATH=tests
python sender.py thedude@thedude.com victim@gmail.com

The result would look like this in victim’s email:

How lamson.html.HtmlMail Works

The rationale behind the HtmlMail class is that you’ll typically have a template you
want to send, some CSS, and then body content that you’ll plug into the template
as a slug for each person. What HtmlMail does is let you setup the template as HTML
with Jinja2, and then specify your CSS using CleverCSS (which is way easier).

When you go to send, you generate a Markdown
template as the body. This lets you write the actual body of your HTML mail the same
way you would a regular email, but still let’s you get a good HTML output using the
HTML/CSS templates. With markdown regular folks can write the marketing copy for
your spam while keeping the design separated.

HtmlMail then glues this all together by doing the following:

  1. Parses your CleverCSS template (after running it through Jinja2).
  2. Runs your content markdown through Jinja2 and markdown2 just like all your other Lamson templates.
  3. Injects your CSS into your HTML tags so that you get your styles even though many clients rip out the style tags from your HTML. See the output sample below.
  4. Knits your generated content into your HTML template as the {{content}} variable.

The end result is that these two lines of code from the above sample:


...
hs = html.HtmlMail(“style.css”, “html_test.html”)

...
msg = hs.respond(locals(), “content.markdown”, From=sys.argv1, To=sys.argv2, Subject=“Test %(title)s”)

Will let you blast out well formatted HTML emails that have a high probability
of displaying in most mail clients.

How The HTML And CSS Looks

Here’s what each of the files in the above sample look like. First the style.css:


body: margin: 10 padding: 20 background: green – 30 color: blue

h1: font-size: 3em h2: font-size: 2em color: yellow h2: font-size: 1em p: padding: 0.3em background: red

h2: color: yellow

#bright: background: black color: white

.dull: background: gray color: black

Notice that CleverCSS supports quite a few
very cool features, like calling functions, nesting, variables, and calculations.

Next you have the outer template html_template.html that wraps your content markdown template. Notice
that there’s no style tag where the above CleverCSS is placed. That gets done
later by lamson.html.HtmlMail.


<html> <head> <title>{{ title }}</title> </head>

<body style=“background: magenta”> <h1 class=“bright”>{{ title }}</h1> {{ content }} <h2 id=“dull”>All done.</h2> </body> </html>

Finally, you have the content.markdown file that has your basic markdown formatted email:


Hello
=

I would love for you to tell me what is going on here joe. NOW!

Alright
———-

This is the best I can come up with.

Zed

You can use any format you want by changing a setting when you construct your HtmlMail object,
but markdown seems to be the closest to what people would send as plain text for their email.
In fact, I’d like to find a way to send the raw markdown as the plaintext version of each HTML
email that goes out.

With the above three components, you then get the following output:

<html>
<head>
<title>Test Message HTML</title>
</head>
<body style=“margin: 10; padding: 20; background: #006200; color: blue”>
<h1 class=“bright” style=“background: black; color: white”>Test Message HTML</h1>
<h1 style=“font-size: 3em”>Hello</h1>
<p style=“padding: 0.3em; background: red”>I would <em>love</em> for you to tell me what is going on here joe.  NOW!</p>
<h2 style=“color: yellow”>Alright</h2>
<p style=“padding: 0.3em; background: red”>This is the best I can come up with.</p>
<p style=“padding: 0.3em; background: red”>Zed</p>
<h2 id=“dull” style=“background: gray; color: black”>All done.</h2>
</body>
</html>

Like magic HtmlMail has taken your CleverCSS file and merged it into the tags of your HTML to
style it, making it work in most clients without forcing you to do it manually. It does this by
walking the CSS and generated HTML with BeautifulSoup
and setting the style attribute as it goes.

Librelist JSON Archives

There’s been a ton of work on librelist.com to make it
work better, but the big feature is a complete JSON dump of the mail archives
and a new Archive Browser for reading the
archives.

The browser is just a fast one day hack I did to prove that the JSON
archive format was good. It uses JQuery UI to build
the interface, and has no servers side software other than Nginx. Yes, you
read that right, there is no Django behind this, just a pure Nginx
setup serving files and directory indexes.

The real meat of this setup is the Lamson server on librelist and an Nginx module.

First, the librelist server generates a JSON version of each archived email which
you can see here with your browser. This JSON actually loads as an
object in JavaScript, so you have code like this:


function appendMessage(msg) { display = '<div id=“message”><div id=“header”>’ + '<div>’ + msg.headers['Date’] + '</div>’ + '<div id=“addressing”>’ + msg.headers['From’] + '</div>’ + '<div id=“subject”>’ + msg.headers['Subject’] + '</div>’ + '</div><div id=“body”><pre>’ + summarize(msg) + '</pre></div></div>’

$(”#messages”).append(display); }

Second, there’s a tiny modification to Nginx’s
autoindex I made so that
it will generate a JSON version of the index instead of HTML
so that you can “browse” the stored JSON files via JavaScript:

function loadDay(date) {
    if(window.MAILING_LIST) {
        $.getJSON(”/archives/” + window.MAILING_LIST + date + “/json/”, {},
        function(data) {
            $(”#messages”).empty();
            base = data0
            msgs = data1
            for(i in msgs) {
                fetchMessage(base + msgs[i])
            }
        });
    } else {
        $(”#messages”).html(”<h2>Now pick a list to look at for that day.</h2>”);
    }
}

With those two components I can export a JSON version of the mailing list archives without
having to run a large web framework.

ChangeLog

For those who are interested, here’s the Bazaar logs from this release, separated into
changes to Lamson and changes to the included librelist.com sample.

Lamson Changes

librelist.com Changes