ELPA

The first thing I do on a fresh Emacs installation is configure the ELPA, or Emacs Lisp Package Archive. If you don’t know what ELPA is, think of it as an apt-get for Emacs (with some quirks though, as discussed below).

You can find the installation instructions here. If everything works, now you should be able to run the command M-x package-list-packages <ENTER>. Read this if you don’t understand how commands and shortcuts work on Emacs.

The list of packages available for installation via ELPA should be displayed in a secondary buffer after a few seconds.

Clojure-Mode

Place the cursor over the line that describes the swank-clojure package, and press I to check it for installation. Do the same thing with paredit. If other packages were checked by accident, move the cursor to the correspondent lines and press U to undo the selection. When you’re done, press X to proceed with the installation.

Then, run the command M-x slime <ENTER>, and Emacs will ask if you want it to download the Clojure JAR files for you. Unfortunately, answering yes won’t do anything.

To fix this, download the Clojure JAR files manually and put them on ~/.clojure/. Then, download the most recent swank-clojure JAR file and put it on ~/.swank-clojure/.

There you go. Restart Emacs and run the command M-x slime <ENTER> to start a Clojure REPL inside Emacs!

Clojure REPL in Emacs

Remember that Slime provides many useful features like auto-completion, docstring lookup, etc. Check the Slime website for more information.

Other Lisps

Run the following command line to install two of the most popular open source implementations of the ANSI Common Lisp standard, CLISP and SBCL:

$ sudo apt-get install clisp sbcl

Finally, open the file ~/.emacs and paste the following code at the end:

;; paredit + lisp modes
(defvar *lisp-modes* '(clojure-mode-hook lisp-mode-hook emacs-lisp-mode-hook
                       lisp-interaction-mode-hook slime-repl-mode-hook))
 
;; config used in all lisp modes
(defun lisp-config ()
  (paredit-mode t)
  (show-paren-mode t))
 
(mapc (lambda (mode)
        (add-hook mode 'lisp-config)) *lisp-modes*)
 
;; other lisp implementations
(eval-after-load "slime"
  '(progn (setq slime-lisp-implementations
                '((sbcl ("/usr/bin/sbcl"))
                  (clisp ("/usr/bin/clisp"))))))

Swank

The latest versions of Slime come with Swank servers for many popular Lisp implementations, although the package available via ELPA looks like a outdated stripped-down version of Slime. Don’t ask me why.

Let’s fix this. Make sure you have a CVS client installed and then run the following command line to get the latest Slime:

$ cvs -d :pserver:anonymous:anonymous@common-lisp.net:/project/slime/cvsroot co slime

Now, go to the newly created directory slime and copy all files from this directory over the files installed via ELPA in ~/.emacs.d/elpa/slime-<VERSION>. Also remove the file slime.elc from the latter directory as this is a compiled version of an old file.

Restart Emacs. The command M-x slime <ENTER> still starts a Clojure REPL. To start a SBCL REPL, for example, run the command M-- M-x slime <ENTER>, type sbcl and press <ENTER>.

SBCL REPL in Emacs

“Jeez, I never saw that many workarounds in a single blog post…”

Everything would be a lot easier if the packages distributed via ELPA were updated more often, so I expect things to get better eventually. I’ll update this post when that happens. (Fingers crossed)

Recently we released Licensator, a simple web application that gathers information on the most popular open source licenses and helps you choose the right license for your own projects.

The language behind Licensator is Clojure, which proved to be great to solve this sort of problem. We’ll see today how the application’s main algorithm works.

REPL: where it all begins

One of the strengths of dynamic languages is how easily we can transform ideas into working code, and Clojure is no exception. In fact, I decided to write Licensator in one of these coding sessions in the REPL, Clojure’s interactive shell. After a few minutes playing with it I saw the problem was indeed easy to solve.

Despite the huge number of licenses, we can easily extract information common to all of them, i.e., if it’s copyleft, or if a licensed work can be used by closed source software, etc.

Let’s take the Apache v2.0 license:

(def *licenses*
     [{:id :apl-v20
       :long-name "Apache License v2.0"
       :short-name "Apache v2.0"
       :url "http://www.opensource.org/licenses/apache2.0.php"
       :copyright true
       :patent true
       :closed-source-linking true
       :derivative-work true
       :affero false
       :copyleft false
       :charge-for-distribution true
       :charge-for-use true
       :compatible-with [:apl-v20 :afl-v30 :cddl-v10 :epl-v10 :freebsd
                         :new-bsd :mit :mpl-v11 :public-domain]}])

You can see the remaining licenses here.

Considering all licenses have the same set of information, how can we find the licenses that are not reciprocal and have explicit copyright terms?

(map :id
  (filter #(and (:copyright %) (not (:copyleft %))) *licenses*))
=> (:afl-v30 :apl-v20 :epl-v10 :freebsd :mit :new-bsd :bsd)

If you know a little bit about functional programming, this code should be easy to follow. In any case, what this code does is return the :id of all licenses where :copyright is true and :copyleft is false.

Another example: which licenses are compatible with Apache v2.0 and CDDL v1.0?

(use 'clojure.set)
 
(map :id
  (filter #(subset? (set [:apl-v20 :cddl-v10])
                    (set (:compatible-with %))) *licenses*))
=> (:apl-v20 :cddl-v10 :epl-v10)

Here we used the clojure.set namespace, which provides a few useful functions to deal with relational algebra. The code is very similar to the previous one; the only difference is that now we used the subset? function to get the licenses compatible with [:apl-v20 :cddl-v10].

One last example: which reciprocal licenses are compatible with Apache v2.0?

(map :id
  (filter #(and (subset? (set [:apl-v20]) (set (:compatible-with %)))
                (:copyleft %)) *licenses*))
=> (:cddl-v10 :agpl-v30 :gpl-v30 :lgpl-v30 :mpl-v11)

At this point we can see that there is no magic behind the algorithm; what it does is to filter the list of licenses based on a function.

The solution

The code we saw up until now works fine, but it’s not appropriate to solve the problem since the matching logic is spread all over the place.

It would be better to express the search criteria as a map object…

(def cmap
     {:patent true, :closed-source-linking true, :compatible-with [:mit :new-bsd]})

…so we need a function to filter the licenses based on that input, like this:

(defn find-matches [cmap data]
  (filter (where cmap) data))

Here we have where, yet to be coded, that returns a function that filters the elements (licenses) in data according to the criteria map cmap.

First things first. Which conditions of cmap match the Apache v2.0 license?

(defn criteria-matches? [license [kval cval]]
  "Checks whether the criteria condition centry is true for license."
  (let [lval (kval license)]
    (if (nil? cval)
      true
      (if (coll? lval)
        (subset? (set cval) (set lval))
        (= cval lval)))))
 
(map (partial criteria-matches? apl-v20) cmap)
=> (true true true)

A license can only be considered compatible with the given criteria if the list only yields true. Therefore, in this case, the Apache v2.0 license is considered to be compatible:

(every? true? (map (partial criteria-matches? apl-v20) cmap))
=> true

Given all this, a possible implementation of the where function might be:

(defn where
  "Returns a function that checks whether all conditions in cmap match for license lentry."
  [cmap]
  (fn [lentry]
    (let [res (map (fn [centry]
		     (let [cval (val centry)
			   lval ((key centry) lentry)]
		       (if (nil? cval)
			 true
			 (if (coll? lval)
			   (subset? (set cval) (set lval))
			   (= cval lval))))) cmap)]
      (every? true? res))))

This code is a combination of the previous two snippets; where gets the criteria map cmap and returns a second function, which in turn gets a license entry lentry and checks whether all criteria conditions match.

Since where is used along with filter in find-matches, all licenses compatible with the criteria map cmap are returned:

(map :id (find-matches cmap *licenses*))
=> (:afl-v30 :apl-v20 :cddl-v10 :epl-v10 :lgpl-v30 :mpl-v11)

What I like about Clojure — and functional programming languages in general — is the ridiculous amount of code it takes to solve certain problems. This algorithm, for example, has only 15 (really short) lines of code. Where is your God now?!

You can see the whole thing in Github.

For me, some tools and distros are particularly useful:

• Clonezilla. Very nice disk/partition cloning tool. It takes some time for beginners to get used to it, but it’s worth the effort. This tool saved my life more than once.
• GParted. Great tool for disk partitioning.
• Linux Mint. Very easy to use Ubuntu-based Linux distro. I particularly like the Live-DVD edition, which includes useful software like VLC, OpenOffice and Java.
• Ubuntu. One of the most popular Linux distros nowadays.
• Memtest86+. Memory testing tool bundled in most Linux distros. Indispensable.

Now, instead of burning one CD/DVD for each one of these tools, we’re going to use one multi-boot USB stick.

Preparing the flash drive

The first step is to install a bootloader on the USB stick, so we can choose which distro/tool to load when booting from it. Lucky for me that my Linux distro of choice, Ubuntu, already comes with Grub2 pre-installed. This particular version of Grub is required since it allows us to mount and boot .ISO images.

Once you have Grub2 installed, plug in your USB stick and make sure that its main partition is bootable and is formated with FAT32 (vfat).

Then, run this command to install Grub2 on the stick:

$ sudo grub-install --no-floppy --root-directory=/media/USB_STICK /dev/sdX

Remember to make root-directory point to the path where the flash drive is mounted, and change /dev/sdX to the actual device. Please double-check the parameters to avoid messing with the wrong disk!

Configuring Grub2

It’s time to download the .ISO images for one or more of the aforementioned distros and put them all inside the directory /boot/iso/ inside flash drive. Then, create the file /boot/grub/grub.cfg with the following content:

menuentry "Linux Mint 9 x86" {
  set isofile="/boot/iso/linuxmint-9-gnome-dvd-i386.iso"
  loopback loop $isofile
  linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile noeject noprompt --
  initrd (loop)/casper/initrd.lz
}
 
menuentry "Ubuntu 10.04 Desktop x86" {
  set isofile="/boot/iso/ubuntu-10.04-desktop-i386.iso"
  loopback loop $isofile
  linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile noeject noprompt --
  initrd (loop)/casper/initrd.lz
}
 
menuentry "Clonezilla Live 1.2.5-17 x86" {
  set isofile="/boot/iso/clonezilla-live-1.2.5-17-i686.iso"
  loopback loop $isofile 
  linux (loop)/live/vmlinuz boot=live union=aufs nolocales noprompt ip=frommedia toram=filesystem.squashfs findiso=$isofile 
  initrd (loop)/live/initrd.img
}
 
menuentry "GParted Live 0.5.2-9 x86" {
  set isofile="/boot/iso/gparted-live-0.5.2-9.iso"
  loopback loop $isofile 
  linux (loop)/live/vmlinuz boot=live union=aufs nolocales noprompt ip=frommedia toram=filesystem.squashfs findiso=$isofile 
  initrd (loop)/live/initrd.img
}
 
menuentry "Memory test (memtest86+)" {
  linux16 /boot/iso/memtest86+.bin
}

Don’t forget to change each isofile variable to match the downloaded images.

Caveat

This stuff is new, and there are only a few distros that can be booted that way. If you want to try this with other distros than the ones listed here, you’d probably need to Google for the right Kernel options to use.

Fortunately, Django is one of the few web frameworks that makes this an easy problem to solve.

How we solved it

A simple way is to create a separate settings module for the mobile version. Doing this, we’re able to run two different webapps while reusing the existing codebase. Also, each version can have its own template directory, URL root configuration, etc.

Let’s do this. In the project root directory, create a file m_settings.py with the following content:

# File: myproject/m_settings.py
 
"""Django settings for the mobile-specific website.
"""
 
# Load the default settings
from settings import *
 
# Domain "m.mydomain.com"
SITE_ID = 2
PREPEND_WWW = False
 
# Used elsewhere to determine which version is running
DEFAULT_VERSION = False
MOBILE_VERSION  = True
 
# URLs for the mobile version
ROOT_URLCONF  = 'myproject.m_urls'
 
# Templates for the mobile version
TEMPLATE_DIRS = (
    os.path.normpath(os.path.dirname(__file__) + '/template/mobile'),
)
 
# Other settings...

These two webapps share the same codebase, but that doesn’t mean they have to use the same URL root configuration. In fact, websites suitable for mobile devices are usually simpler, with fewer URL mappings.

To define separate URL root configuration for the mobile version, create a file m_urls.py with the following content:

# File: myproject/m_urls.py
 
"""Mobile-specific URL definition.
"""
 
from django.conf import settings
from django.conf.urls.defaults import *
# from django.contrib import admin
 
# admin.autodiscover()
 
# Error page handlers
handler404 = 'website.views.error_404_handler'
handler500 = 'website.views.error_500_handler'
 
# Root URL patterns
urlpatterns = patterns('',
    # I18N
    (r'^i18n/', include('django.conf.urls.i18n')),
 
    # Internal apps
    (r'^contact/', include('contact.urls')),
 
    # (r'^admin/doc/', include('django.contrib.admindocs.urls')),
    # (r'^admin/', include(admin.site.urls)),
 
    # Other settings...
)

Note that, in this particular example, the Admin app isn’t accessible from mobile version. This just shows that we can structure each webapp differently without too much trouble.

Developing

You already know how to run the development server for the default version. Most people don’t know, however, that it’s possible to choose a different port number and settings module, which allows us to launch one development server for each webapp version:

# Terminal window 1: default version
$ python manage.py runserver 8000
 
# Terminal window 2: mobile version
$ python manage.py runserver --settings=m_settings 8001

Just develop your stuff and see the results in both versions right away. Easy peasy.

Testing

You probably noticed that we have two settings that can be used to identify what webapp version is being run, DEFAULT_VERSION and MOBILE_VERSION.

So it’s just a matter of checking those settings when you need to do different things in each webapp version. This is particularly useful when you use an app in both versions and want to create separate tests for each version:

# File: myproject/myapp/tests/__init__.py
 
"""Unit tests module for MyApp.
"""
 
from django.conf import settings
 
if getattr(settings, 'DEFAULT_VERSION', False):
    # Tests the default version
    from default_models_tests import *
    from default_navigation_tests import *
 
if getattr(settings, 'MOBILE_VERSION', False):
    # Tests the mobile version
    from mobile_models_tests import *
    from mobile_navigation_tests import *

We can run the tests for each version in a similar way we did before with the runserver command:

# Default version
$ python manage.py test [appname...]
 
# Mobile version
$ python manage.py test --settings=m_settings [appname...]

Deploying

Despite what you may think, this is the easy part.

Since deploying Django with Apache and mod_wsgi is the recommended way to get Django into production, it’s likely that you already have a WSGI file like this:

import os, sys
 
APPS_PATH   = '/home/user/wsgi_apps'
MYPROJECT_PATH = '%s/myproject' % APPS_PATH
 
# Third-party eggs
import site
site.addsitedir('/home/user/.python/lib')
 
sys.path += [APPS_PATH, MYPROJECT_PATH]
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'
 
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

All we need to do is configure another WSGI-enabled subdomain on our server, e.g. m.mywebsite.com, that refers to another WSGI file like this one:

import os, sys
 
APPS_PATH   = '/home/user/wsgi_apps'
MYPROJECT_PATH = '%s/myproject' % APPS_PATH
 
# Third-party eggs
import site
site.addsitedir('/home/user/.python/lib')
 
sys.path += [APPS_PATH, MYPROJECT_PATH]
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.m_settings' # This line
 
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

The only difference between these two WSGI files is that each one points to a different settings module. And no, you don’t need to have a separate copy for each version. We are using the same codebase for both versions, remember?

Minimizing the traffic

Amazingly, not everyone has access ultra-fast internet connections on their devices, so it’s very important to keep your sites as lightweight as possible.

There are several open source projects that try to address this problem. One of them is django-compress, a Django app that provides an automated system for compressing CSS and JavaScript files.

In fact, doesn’t matter if your websites target mobile devices or not; you should always try to minimize the amount of traffic.

What’s left?

There’s one small problem though: when someone visit the regular website from a mobile device, or vice-versa, they are not redirected to the appropriate version. But this is a topic for another article!

In this article I’ll show you how to configure Exim to send messages through Gmail.

Installing Exim

First we need to install Exim and its dependencies, which is a simple task in modern Linux distros like Ubuntu/Debian:

$ sudo apt-get install exim4

Configuring Exim as a relay server

There’s a wizard that helps us with the basic setup; run the following command line to open it and fill each screen as follows:

$ sudo dpkg-reconfigure exim4-config

1. General type of mail configuration: mail sent by smarthost; received via SMTP or fetchmail
2. System mail name: yourdomain.com
3. IP-addresses to listen on for incoming connections: 127.0.0.1 ; ::1
4. Other destinations for which mail is accepted: Leave it blank
5. Machines to relay mail for: Leave it blank
6. Host name for the outgoing smarthost: smtp.gmail.com:587
7. Hide local mail name in outgoing mail: Yes
8. Visible domain name for local users: yourdomain.com
9. Keep number of DNS-queries minimal: No
10. Delivery method for local mail: mbox format in /var/mail/
11. Split configuration into small files: Yes
12. Root and postmaster mail recipient: Leave it blank

Now we have to configure the user credentials in the file /etc/exim4/passwd.client by adding the following line:

# Replace [username] and [password] by your account's credentials
*.google.com:[username]:[password]

Let’s finish the configuration by running the following command line:

$ sudo update-exim4.conf

Testing the setup

Here’s an old-school way to test if our setup is working — with telnet command:

$ telnet localhost 25
    Trying ::1...
    Connected to localhost.
    Escape character is '^]'.
    220 localhost ESMTP Exim 4.71 Wed, 23 Jun 2010 22:28:56 -0300
HELO localhost
    250 localhost Hello localhost [::1]
MAIL FROM: username@yourdomain.com
    250 OK
RCPT TO: destination@email.com
    250 Accepted
DATA
    354 Enter message, ending with "." on a line by itself
SUBJECT: Message subject
Testing Exim via telnet. w00t!
.
    250 OK id=1ORbG5-0005uh-EF
QUIT
    221 localhost closing connection
    Connection closed by foreign host.

Open the log files at /var/log/exim4 to see if the message relayed correctly. This is how a successful attempt looks like:

2010-06-23 22:29:47 1ORbG5-0005uh-EF <= local@yourdomain.com H=localhost [::1] P=smtp S=345
2010-06-23 22:29:50 1ORbG5-0005uh-EF => destination@email.com R=smarthost T=remote_smtp_smarthost H=gmail-smtp-msa.l.google.com [74.125.45.109] ..."
2010-06-23 22:29:50 1ORbG5-0005uh-EF Completed

If the log looks good and you didn’t get the message, it’s a good idea to check the spam box of the destination email before you change any configuration.

And with the World Cup comes the sweepstakes. Lots of.

Although the rules might change from one place to another, these are the ones we use the most here in Brazil:

1. Everyone try to predict the score of a particular match, paying a fixed amount for each guess (eg $1.00).
2. At the end, the ammount collected is divided equally among the winners. If there are no winners: (a) the sweepstake “accumulates”¹, or (b) everyone get their money back, or (c) you-crazy-rule-here.

I like this kind of sweepstake, but it could be better in some respects.

First, not everybody has the habit of carrying small bills around, so the sweepstakes manager (a.k.a. YOU) loses a lot of time chasing change. Besides, I think the fixed-valued bet is kind of boring since the amount collected is divided equally among the winners. I think things get more exciting when people can choose how much to pay for each guess, so the amount collected is divided proportionally among the winners. In this case, when several people win, earns more who bets more.

Since I don’t like to do the math myself, here’s the challenge: create a program that receives a list with all guesses and returns the list of winners along with the proportional amount to be paid to each one of them.

So, shall we?

Solving the problem… with Clojure

Write more code than necessary is going out of style these days, so I decided to code a solution in Clojure, a functional programming language that makes this sort of “problem” really easy to solve. Just launch the REPL and profit!

The first thing to do is define how to represent each guess:

(defstruct bet :name :amount :guess :rate)

All guesses has the same information. That’s why we defined a StructMap with the keys :name, :amount, :guess, and :rate.

A list of possible guesses might be:

(def pool
     [(struct bet "Daniel" 2.0 [2 0]) ;; Daniel bets $2.00 on score 2x0
      (struct bet "John"   4.0 [0 0]) ;; John bets $4.00 on score 0x0
      (struct bet "Maria"  1.0 [2 0]) ;; Maria bets $1.00 on score 2x0
     ])

Okay, we already know how each guess is represented. Now, how to find the sweepstake’s total amount? Simple, all we have to do is to sum the amount of all guesses:

(defn pool-amount [poolseq]
  (reduce + (map :amount poolseq)))
 
;; testing
(pool-amount pool)
=> 7.0

Great, but how much each guess represent in relation to the whole, that is, what’s the proportion of each guess? All we have to do is to divide the amount of each guess by the sweepstake’s total amount:

(defn pool-rates [poolseq]
  (let [sum (pool-amount poolseq)]
    (map #(assoc % :rate (/ (:amount %) sum)) poolseq)))
 
;; testing
(map :rate (pool-rates pool))
=> (0.285 ..., 0.571 ..., 0.142 ...)
 
(reduce + (map :rate (pool-rates pool)))
=> 1.0

Note that, if everyone wins, the amount earned by John would be the double of the amount earned by Daniel, which would be the double of the amount earned by Maria. Justice, finally!

To finish the code, we just need a function that filters the winners and calculates how much to pay to each winner:

(defn pool-winners [poolseq result]
  (let [sum (pool-amount poolseq)
        winners (filter #(= (:guess %) result) poolseq)]
    (map #(assoc % :amount (double (* (:rate %) sum))) (pool-rates winners))))
 
;; testing
(pool-winners pool [1 0]) ;; no winners
=> ()
 
(pool-winners pool [0 0]) ;; just one winner
=> ({:name "John", :amount 7.0, :guess [0 0], :rate 1.0})
 
(pool-winners pool [2 0]) ;; several winners
=> ({:name "Daniel", :amount 4.66 ..., :guess [2 0], :rate 0.66 ...}
    {:name "Maria", :amount 2.33 ..., :guess [2 0], :rate 0.33 ...})

A major advantage of this solution is that it allows you to bet on anything. Want to bet on the sex of a baby shortly after the good news (or bad news, whatever)?

(def pool [(struct bet "Daniel" 3.0 :male)
           (struct bet "John"   5.0 :female)
           (struct bet "Maria"  1.0 :female)])
 
(pool-winners pool :female)
=> ({:name "John", :amount 7.5, ...} {:name "Maria", :amount 1.5, ...})

There you go, a simple solution for a simple problem.

The code

Although it’s possible to solve this problem in Lisp/Clojure with a one-liner (the Reader works with data structures after all), I decided to write a code that is as readable as possible for those who are starting with Clojure:

(defstruct bet :name :amount :guess :rate)
 
(defn pool-amount [poolseq]
  (reduce + (map :amount poolseq)))
 
(defn pool-rates [poolseq]
  (let [sum (pool-amount poolseq)]
    (map #(assoc % :rate (/ (:amount %) sum)) poolseq)))
 
(defn pool-winners [poolseq result]
  (let [sum (pool-amount poolseq)
        winners (filter #(= (:guess %) result) poolseq)]
    (map #(assoc % :amount (double (* (:rate %) sum))) (pool-rates winners))))

Your turn

Do you program in Haskell? Python? Ruby? Brainfuck? How would you solve such problem in your favorite programming language?

Update (Nov 3, 2010). Common Lisp solution, using copy-* to avoid changes in state:

(defstruct bet name amount guess rate)
 
(defun pool-amount (pool)
  (reduce #'+ (mapcar #'bet-amount pool)))
 
(defun pool-rates (pool)
  (let ((sum (pool-amount pool)))
    (mapcar (lambda (bet)
              (let ((x (copy-bet bet)))
                (setf (bet-rate x) (/ (bet-amount x) sum))
                x)) pool)))
 
(defun pool-winners (pool result)
  (let ((sum (pool-amount pool))
        (winners (loop for bet in pool when (equal (bet-guess bet) result) collect bet)))
    (mapcar (lambda (winner)
              (let ((x (copy-bet winner)))
                (setf (bet-amount x) (* (bet-rate x) sum))
                x)) (pool-rates winners))))

Yes, I said Django 1.2 comes with built-in user “messages” framework. Great, isn’t it? Django really needed a built-in contrib app to solve this problem. This doesn’t mean, however, that Django-Flash will be discontinued or abandoned.

So, if you use Django-Flash in your projects, don’t worry! Nothing will change. Django-Flash is already compatible with Django 1.2, and we’ll keep improving Django-Flash for a number of reasons.

First of all, we care about the people who use our software. We won’t break your stuff if you decide to upgrade to a newer version of Django – or if you decide to keep using the previous version.

Also, freedom of choice is a good thing. You know, there’s not only one true way to solve every problem. People should be free to choose whatever tool they think is right in a given situation.

For example, based on my first impressions, there are a couple of things I didn’t like at all about the new user “messages” framework:

1. It’s verbose even for the simple cases
2. It ties messages to priority numbers, encouraging you to use it like a logging/debugging framework. I mean, a user “messages” framework is not a logging/debugging framework, right?

But that’s just my opinion. I guess this is what makes open source so great!

I ended up with a Nokia 5800 XpressMusic, a relatively cheap smartphone with a nice feature set. Specifically, what took me for granted was the ability to run Python applications in it!

But what exactly can we do with Python for S60? Well, to answer that question, I decided to implement a non-trivial app to see exactly what it has to offer. The app in question is a Bluetooth remote control called Pytriloquist. (see also screenshots and a video demonstration.)

What’s Good

Platform Independence. The S60 SDK is for Windows only (what a surprise). This is obviously bad for those who use other operating systems. Fortunately, you don’t need a massive SDK to develop and deploy S60 apps if you write them in Python; all you need is Python for S60, a text editor, and a device for testing.

Language and Standard Library. Python is a terrific language: compact, beautiful, powerful, and have an amazing standard library. The good news is that the most popular modules work great on S60. For example, it’s even possible to localize your apps using the gettext module!

What’s Bad

Work in Progress. Python for S60, as long as I know, is a work in progress. This means the language itself works fine, but you can expect missing modules or some bugs in standard library. For example, when writing the Bluetooth client code for my app, I encountered a very annoying bug in which you can only call btsocket.bt_discover() once!

Multi-Module Code. If you are testing your app with Python ScriptShell, everything will work out-of-the-box as long as keep your whole code inside a single massive module. If you break up your code into several modules (or worse, packages) you won’t be able to have modules importing each other unless you adjust the PYTHON_PATH accordingly.

Incomplete and Outdated Documentation. There are some PDFs and API documentation pages available, but they are incomplete and outdated, and this is a bad thing when it comes to dynamic languages; we don’t have a compiler to tell us what’s wrong with our code. This is why Python programmers are so careful about documentation in the first place.

What’s Ugly

Limited Access to Native Features. Specially regarding the native UI system: you cannot change the label of the two standard soft keys, nor add other soft keys, nor add more than two tabs in a tabbed UI, nor have an empty Listbox, nor lots of other things. You can write simple UIs without much effort, that’s true, but those shortcomings just make Python for S60 unusable for more complex projects.

Project Inactivity. The project appears to be on hold. New bug reports are submitted every now and then, but no bugs are closed for six months.

Conclusion

Let’s hope things change for the better after the last announcement made by the Symbian Foundation, but in the meantime, if you pretend to create serious apps for S60, do it in C++.

This is a fairly common thing, at least for the companies I worked for so far, and that’s what scares me the most. Hopefully, you can easily make things better if you want to.

In this post, I’ll show you a few techniques that can be used to write clean JSPs without resorting to any kind of templating system whatsoever.

An example

The first step is to define the base template for the application pages. Consider the following XHTML example and its graphic representation:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta http-equiv="content-type" content="text/html; charset=UTF-8"/>
        <meta name="keywords" content="keyword1, keyword2, keyword3" />
        <meta name="description" content="Page description" />
        <title>Page title</title>
 
        <!-- Essential scripts -->
        <link rel="stylesheet" type="text/css" href="css/style.css"/>
        <script type="text/javascript" src="js/query.js"></script>
 
        <!-- Extra header -->
        <script type="text/javascript" src="js/util.js"></script>
    </head>
    <body>
        <!-- Main -->
        <div id="wrapper">
            <div id="header"></div>
            <div id="content"></div>
            <div id="footer"></div>
        </div> <!-- Main -->
 
        <script type="text/javascript">
            // Other scripts (e.g. Google Analytics)
        </script>
    </body>
</html>

Template graphic representation

What usually happens, even with a simple template such as this, is a high level of code duplication among the different pages due to the heavy use of Copy and Paste Programming:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta http-equiv="content-type" content="text/html; charset=UTF-8"/>
        <meta name="keywords"    content="keyword1, keyword2, keyword3" />
        <meta name="description" content="Page description" />
        <title>Page title</title>
 
        <!-- Essential scripts -->
        <jsp:include page="scripts.jsp"/>
 
        <!-- Extra header -->
        <script type="text/javascript" src="js/util.js"></script>
    </head>
    <body>
        <!-- Main -->
        <div id="wrapper">
            <jsp:include file="header.jsp"/>
            <div id="content">Page content here</div>
            <jsp:include page="footer.jsp"/>
        </div> <!-- Main -->
 
        <script type="text/javascript">
            // Other scripts
        </script>
    </body>
</html>

See? Why break the template into small pieces if you are going keep duplicating code anyway? The best thing to do is to translate that template into something like this:

<%@ taglib tagdir="/WEB-INF/tags/layout" prefix="layout" %>
<layout:page title="Title" description="Page description" keywords="keyword1, keyword2, keyword3">
    <jsp:attribute name="extraHeader">
        <script type="text/javascript" src="js/util.js"></script>
    </jsp:attribute>
    <jsp:attribute name="extraBottom">
        // Other scripts (e.g. Google Analytics)
    </jsp:attribute>
    <jsp:body>
        Page content here
    </jsp:body>
</layout:page>

Content reuse with tag files

There are three types of JSP tags, each with its own strengths and weaknesses: classic tag handlers, simple tag handlers and tag files. Today I’ll cover the basics of tag files, but every Java developer should learn when and how to use the other two.

The first thing we need to do is create the WEB-INF/tags/layout directory, which will keep the tag files we are about to create. Now, create two files inside this directory, one for the header and the other for the footer:

header.tag

<%@ tag body-content="empty" description="Header tag file" %>
<div id="header">
    <!-- Header section here -->
</div>

footer.tag

<%@ tag body-content="empty" description="Footer tag file" %>
<div id="footer">
    <!-- Footer section here -->
</div>

As we can see, tag files are just like JSPs, but with some special properties.

The last tag file, page.tag, defines the overall page structure:

<%@ tag description="Page layout" %>
 
<%@ attribute name="title"       required="true" description="Page title" %>
<%@ attribute name="keywords"    required="true" description="Page keywords to improve SEO" %>
<%@ attribute name="description" required="true" description="Page description" %>
 
<%@ attribute name="extraHeader" fragment="true" description="Extra code to put before </head>" %>
<%@ attribute name="extraBottom" fragment="true" description="Extra code to put before </body>" %>
 
<%@ taglib tagdir="/WEB-INF/tags/layout" prefix="layout" %>
 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta http-equiv="content-type" content="text/html; charset=UTF-8"/>
 
        <!-- Displaying the attributes using EL -->
        <meta name="keywords"    content="${keywords}" />
        <meta name="description" content="${description}" />
        <title>${title}</title>
 
        <!-- Essential scripts -->
        <link rel="stylesheet" type="text/css" href="css/style.css"/>
        <script type="text/javascript" src="js/query.js"></script>
 
        <!-- Process the given input fragment -->
        <jsp:invoke fragment="extraHeader"/>
    </head>
    <body>
        <!-- Main -->
        <div id="wrapper">
            <!-- Renders the page header -->
            <layout:header/>
 
            <!-- Renders the tag body inside a DIV -->
            <div id="content"><jsp:doBody/></div>
 
            <!-- Renders the page footer -->
            <layout:footer/>
        </div> <!-- Main -->
 
        <!-- Process the given input fragment -->
        <jsp:invoke fragment="extraBottom"/>
    </body>
</html>

This last tag file is more complex than the other two, because you can pass to it both values and fragments. Besides, this tag accepts a body, which gets executed with <jsp:doBody/>.

Of course, the example shown here is fairly simple, so we didn’t have to use more advanced features like variables and dynamic attributes. Be that as it may, it’s a good idea to know what the technology has to offer. You’ll never know when a feature will be useful until you need it.

Usage

Now, let’s put those tag files to use:

<%@ taglib tagdir="/WEB-INF/tags/layout" prefix="layout" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<layout:page title="Index page" description="This is the index page of my amazing app" keywords="amazing, app">
    <jsp:attribute name="extraHeader">
        <script type="text/javascript" src="js/util.js"></script>
        <script type="text/javascript" src="js/some_jquery_plugin.js"></script>
    </jsp:attribute>
    <jsp:attribute name="extraBottom">
        $(function() {
            // Call some JQuery function here, specific for this page
        });
    </jsp:attribute>
    <jsp:body>
        Welcome to my amazing app!<br/>
        1+1 = ${1+1}
 
        <ul>
        <c:forEach items="${objects}" var="obj">
            <li>${obj.attr}</li>
        </c:forEach>
        </ul>
    </jsp:body>
</layout:page>

Although the application as a whole follow the same visual standard, each page may need additional resources (JavaScript, CSS, etc.). This is hardly a problem because our template allows us to include extra content if necessary.

The other advantage is that tag files allow the use of custom tags and unified EL, but deny the use of scriptlets, which is a good thing if you want to get rid of messy JSPs.

Recommended reading

• The Java EE 5 Tutorial;
• Head First Servlets & JSP;

All Python lovers out there will probably notice that the website was created with Sphinx, a very popular documentation tool in the Python world.

This is very interesting by itself; Sphinx can be used to document softwares written in a wide range of programming languages, not just Python! In fact, our experience using it to document a JavaScript project couldn’t be better.

Of course there’s a lot of room for improvements, but we think we’re getting there. Hopefully.