Recently we released Licensator, a simple web application that gathers information on the most popular open source licenses and helps you choose the right license for your own projects.

The language behind Licensator is Clojure, which proved to be great to solve this sort of problem. We’ll see today how the application’s main algorithm works.

REPL: where it all begins

One of the strengths of dynamic languages is how easily we can transform ideas into working code, and Clojure is no exception. In fact, I decided to write Licensator in one of these coding sessions in the REPL, Clojure’s interactive shell. After a few minutes playing with it I saw the problem was indeed easy to solve.

Despite the huge number of licenses, we can easily extract information common to all of them, i.e., if it’s copyleft, or if a licensed work can be used by closed source software, etc.

Let’s take the Apache v2.0 license:

(def *licenses*
     [{:id :apl-v20
       :long-name "Apache License v2.0"
       :short-name "Apache v2.0"
       :url "http://www.opensource.org/licenses/apache2.0.php"
       :copyright true
       :patent true
       :closed-source-linking true
       :derivative-work true
       :affero false
       :copyleft false
       :charge-for-distribution true
       :charge-for-use true
       :compatible-with [:apl-v20 :afl-v30 :cddl-v10 :epl-v10 :freebsd
                         :new-bsd :mit :mpl-v11 :public-domain]}])

You can see the remaining licenses here.

Considering all licenses have the same set of information, how can we find the licenses that are not reciprocal and have explicit copyright terms?

(map :id
  (filter #(and (:copyright %) (not (:copyleft %))) *licenses*))
=> (:afl-v30 :apl-v20 :epl-v10 :freebsd :mit :new-bsd :bsd)

If you know a little bit about functional programming, this code should be easy to follow. In any case, what this code does is return the :id of all licenses where :copyright is true and :copyleft is false.

Another example: which licenses are compatible with Apache v2.0 and CDDL v1.0?

(use 'clojure.set)
 
(map :id
  (filter #(subset? (set [:apl-v20 :cddl-v10])
                    (set (:compatible-with %))) *licenses*))
=> (:apl-v20 :cddl-v10 :epl-v10)

Here we used the clojure.set namespace, which provides a few useful functions to deal with relational algebra. The code is very similar to the previous one; the only difference is that now we used the subset? function to get the licenses compatible with [:apl-v20 :cddl-v10].

One last example: which reciprocal licenses are compatible with Apache v2.0?

(map :id
  (filter #(and (subset? (set [:apl-v20]) (set (:compatible-with %)))
                (:copyleft %)) *licenses*))
=> (:cddl-v10 :agpl-v30 :gpl-v30 :lgpl-v30 :mpl-v11)

At this point we can see that there is no magic behind the algorithm; what it does is to filter the list of licenses based on a function.

The solution

The code we saw up until now works fine, but it’s not appropriate to solve the problem since the matching logic is spread all over the place.

It would be better to express the search criteria as a map object…

(def cmap
     {:patent true, :closed-source-linking true, :compatible-with [:mit :new-bsd]})

…so we need a function to filter the licenses based on that input, like this:

(defn find-matches [cmap data]
  (filter (where cmap) data))

Here we have where, yet to be coded, that returns a function that filters the elements (licenses) in data according to the criteria map cmap.

First things first. Which conditions of cmap match the Apache v2.0 license?

(defn criteria-matches? [license [kval cval]]
  "Checks whether the criteria condition centry is true for license."
  (let [lval (kval license)]
    (if (nil? cval)
      true
      (if (coll? lval)
        (subset? (set cval) (set lval))
        (= cval lval)))))
 
(map (partial criteria-matches? apl-v20) cmap)
=> (true true true)

A license can only be considered compatible with the given criteria if the list only yields true. Therefore, in this case, the Apache v2.0 license is considered to be compatible:

(every? true? (map (partial criteria-matches? apl-v20) cmap))
=> true

Given all this, a possible implementation of the where function might be:

(defn where
  "Returns a function that checks whether all conditions in cmap match for license lentry."
  [cmap]
  (fn [lentry]
    (let [res (map (fn [centry]
		     (let [cval (val centry)
			   lval ((key centry) lentry)]
		       (if (nil? cval)
			 true
			 (if (coll? lval)
			   (subset? (set cval) (set lval))
			   (= cval lval))))) cmap)]
      (every? true? res))))

This code is a combination of the previous two snippets; where gets the criteria map cmap and returns a second function, which in turn gets a license entry lentry and checks whether all criteria conditions match.

Since where is used along with filter in find-matches, all licenses compatible with the criteria map cmap are returned:

(map :id (find-matches cmap *licenses*))
=> (:afl-v30 :apl-v20 :cddl-v10 :epl-v10 :lgpl-v30 :mpl-v11)

What I like about Clojure — and functional programming languages in general — is the ridiculous amount of code it takes to solve certain problems. This algorithm, for example, has only 15 (really short) lines of code. Where is your God now?!

You can see the whole thing in Github.

Fortunately, Django is one of the few web frameworks that makes this an easy problem to solve.

How we solved it

A simple way is to create a separate settings module for the mobile version. Doing this, we’re able to run two different webapps while reusing the existing codebase. Also, each version can have its own template directory, URL root configuration, etc.

Let’s do this. In the project root directory, create a file m_settings.py with the following content:

# File: myproject/m_settings.py
 
"""Django settings for the mobile-specific website.
"""
 
# Load the default settings
from settings import *
 
# Domain "m.mydomain.com"
SITE_ID = 2
PREPEND_WWW = False
 
# Used elsewhere to determine which version is running
DEFAULT_VERSION = False
MOBILE_VERSION  = True
 
# URLs for the mobile version
ROOT_URLCONF  = 'myproject.m_urls'
 
# Templates for the mobile version
TEMPLATE_DIRS = (
    os.path.normpath(os.path.dirname(__file__) + '/template/mobile'),
)
 
# Other settings...

These two webapps share the same codebase, but that doesn’t mean they have to use the same URL root configuration. In fact, websites suitable for mobile devices are usually simpler, with fewer URL mappings.

To define separate URL root configuration for the mobile version, create a file m_urls.py with the following content:

# File: myproject/m_urls.py
 
"""Mobile-specific URL definition.
"""
 
from django.conf import settings
from django.conf.urls.defaults import *
# from django.contrib import admin
 
# admin.autodiscover()
 
# Error page handlers
handler404 = 'website.views.error_404_handler'
handler500 = 'website.views.error_500_handler'
 
# Root URL patterns
urlpatterns = patterns('',
    # I18N
    (r'^i18n/', include('django.conf.urls.i18n')),
 
    # Internal apps
    (r'^contact/', include('contact.urls')),
 
    # (r'^admin/doc/', include('django.contrib.admindocs.urls')),
    # (r'^admin/', include(admin.site.urls)),
 
    # Other settings...
)

Note that, in this particular example, the Admin app isn’t accessible from mobile version. This just shows that we can structure each webapp differently without too much trouble.

Developing

You already know how to run the development server for the default version. Most people don’t know, however, that it’s possible to choose a different port number and settings module, which allows us to launch one development server for each webapp version:

# Terminal window 1: default version
$ python manage.py runserver 8000
 
# Terminal window 2: mobile version
$ python manage.py runserver --settings=m_settings 8001

Just develop your stuff and see the results in both versions right away. Easy peasy.

Testing

You probably noticed that we have two settings that can be used to identify what webapp version is being run, DEFAULT_VERSION and MOBILE_VERSION.

So it’s just a matter of checking those settings when you need to do different things in each webapp version. This is particularly useful when you use an app in both versions and want to create separate tests for each version:

# File: myproject/myapp/tests/__init__.py
 
"""Unit tests module for MyApp.
"""
 
from django.conf import settings
 
if getattr(settings, 'DEFAULT_VERSION', False):
    # Tests the default version
    from default_models_tests import *
    from default_navigation_tests import *
 
if getattr(settings, 'MOBILE_VERSION', False):
    # Tests the mobile version
    from mobile_models_tests import *
    from mobile_navigation_tests import *

We can run the tests for each version in a similar way we did before with the runserver command:

# Default version
$ python manage.py test [appname...]
 
# Mobile version
$ python manage.py test --settings=m_settings [appname...]

Deploying

Despite what you may think, this is the easy part.

Since deploying Django with Apache and mod_wsgi is the recommended way to get Django into production, it’s likely that you already have a WSGI file like this:

import os, sys
 
APPS_PATH   = '/home/user/wsgi_apps'
MYPROJECT_PATH = '%s/myproject' % APPS_PATH
 
# Third-party eggs
import site
site.addsitedir('/home/user/.python/lib')
 
sys.path += [APPS_PATH, MYPROJECT_PATH]
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'
 
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

All we need to do is configure another WSGI-enabled subdomain on our server, e.g. m.mywebsite.com, that refers to another WSGI file like this one:

import os, sys
 
APPS_PATH   = '/home/user/wsgi_apps'
MYPROJECT_PATH = '%s/myproject' % APPS_PATH
 
# Third-party eggs
import site
site.addsitedir('/home/user/.python/lib')
 
sys.path += [APPS_PATH, MYPROJECT_PATH]
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.m_settings' # This line
 
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

The only difference between these two WSGI files is that each one points to a different settings module. And no, you don’t need to have a separate copy for each version. We are using the same codebase for both versions, remember?

Minimizing the traffic

Amazingly, not everyone has access ultra-fast internet connections on their devices, so it’s very important to keep your sites as lightweight as possible.

There are several open source projects that try to address this problem. One of them is django-compress, a Django app that provides an automated system for compressing CSS and JavaScript files.

In fact, doesn’t matter if your websites target mobile devices or not; you should always try to minimize the amount of traffic.

What’s left?

There’s one small problem though: when someone visit the regular website from a mobile device, or vice-versa, they are not redirected to the appropriate version. But this is a topic for another article!

And with the World Cup comes the sweepstakes. Lots of.

Although the rules might change from one place to another, these are the ones we use the most here in Brazil:

1. Everyone try to predict the score of a particular match, paying a fixed amount for each guess (eg $1.00).
2. At the end, the ammount collected is divided equally among the winners. If there are no winners: (a) the sweepstake “accumulates”¹, or (b) everyone get their money back, or (c) you-crazy-rule-here.

I like this kind of sweepstake, but it could be better in some respects.

First, not everybody has the habit of carrying small bills around, so the sweepstakes manager (a.k.a. YOU) loses a lot of time chasing change. Besides, I think the fixed-valued bet is kind of boring since the amount collected is divided equally among the winners. I think things get more exciting when people can choose how much to pay for each guess, so the amount collected is divided proportionally among the winners. In this case, when several people win, earns more who bets more.

Since I don’t like to do the math myself, here’s the challenge: create a program that receives a list with all guesses and returns the list of winners along with the proportional amount to be paid to each one of them.

So, shall we?

Solving the problem… with Clojure

Write more code than necessary is going out of style these days, so I decided to code a solution in Clojure, a functional programming language that makes this sort of “problem” really easy to solve. Just launch the REPL and profit!

The first thing to do is define how to represent each guess:

(defstruct bet :name :amount :guess :rate)

All guesses has the same information. That’s why we defined a StructMap with the keys :name, :amount, :guess, and :rate.

A list of possible guesses might be:

(def pool
     [(struct bet "Daniel" 2.0 [2 0]) ;; Daniel bets $2.00 on score 2x0
      (struct bet "John"   4.0 [0 0]) ;; John bets $4.00 on score 0x0
      (struct bet "Maria"  1.0 [2 0]) ;; Maria bets $1.00 on score 2x0
     ])

Okay, we already know how each guess is represented. Now, how to find the sweepstake’s total amount? Simple, all we have to do is to sum the amount of all guesses:

(defn pool-amount [poolseq]
  (reduce + (map :amount poolseq)))
 
;; testing
(pool-amount pool)
=> 7.0

Great, but how much each guess represent in relation to the whole, that is, what’s the proportion of each guess? All we have to do is to divide the amount of each guess by the sweepstake’s total amount:

(defn pool-rates [poolseq]
  (let [sum (pool-amount poolseq)]
    (map #(assoc % :rate (/ (:amount %) sum)) poolseq)))
 
;; testing
(map :rate (pool-rates pool))
=> (0.285 ..., 0.571 ..., 0.142 ...)
 
(reduce + (map :rate (pool-rates pool)))
=> 1.0

Note that, if everyone wins, the amount earned by John would be the double of the amount earned by Daniel, which would be the double of the amount earned by Maria. Justice, finally!

To finish the code, we just need a function that filters the winners and calculates how much to pay to each winner:

(defn pool-winners [poolseq result]
  (let [sum (pool-amount poolseq)
        winners (filter #(= (:guess %) result) poolseq)]
    (map #(assoc % :amount (double (* (:rate %) sum))) (pool-rates winners))))
 
;; testing
(pool-winners pool [1 0]) ;; no winners
=> ()
 
(pool-winners pool [0 0]) ;; just one winner
=> ({:name "John", :amount 7.0, :guess [0 0], :rate 1.0})
 
(pool-winners pool [2 0]) ;; several winners
=> ({:name "Daniel", :amount 4.66 ..., :guess [2 0], :rate 0.66 ...}
    {:name "Maria", :amount 2.33 ..., :guess [2 0], :rate 0.33 ...})

A major advantage of this solution is that it allows you to bet on anything. Want to bet on the sex of a baby shortly after the good news (or bad news, whatever)?

(def pool [(struct bet "Daniel" 3.0 :male)
           (struct bet "John"   5.0 :female)
           (struct bet "Maria"  1.0 :female)])
 
(pool-winners pool :female)
=> ({:name "John", :amount 7.5, ...} {:name "Maria", :amount 1.5, ...})

There you go, a simple solution for a simple problem.

The code

Although it’s possible to solve this problem in Lisp/Clojure with a one-liner (the Reader works with data structures after all), I decided to write a code that is as readable as possible for those who are starting with Clojure:

(defstruct bet :name :amount :guess :rate)
 
(defn pool-amount [poolseq]
  (reduce + (map :amount poolseq)))
 
(defn pool-rates [poolseq]
  (let [sum (pool-amount poolseq)]
    (map #(assoc % :rate (/ (:amount %) sum)) poolseq)))
 
(defn pool-winners [poolseq result]
  (let [sum (pool-amount poolseq)
        winners (filter #(= (:guess %) result) poolseq)]
    (map #(assoc % :amount (double (* (:rate %) sum))) (pool-rates winners))))

Your turn

Do you program in Haskell? Python? Ruby? Brainfuck? How would you solve such problem in your favorite programming language?

Update (Nov 3, 2010). Common Lisp solution, using copy-* to avoid changes in state:

(defstruct bet name amount guess rate)
 
(defun pool-amount (pool)
  (reduce #'+ (mapcar #'bet-amount pool)))
 
(defun pool-rates (pool)
  (let ((sum (pool-amount pool)))
    (mapcar (lambda (bet)
              (let ((x (copy-bet bet)))
                (setf (bet-rate x) (/ (bet-amount x) sum))
                x)) pool)))
 
(defun pool-winners (pool result)
  (let ((sum (pool-amount pool))
        (winners (loop for bet in pool when (equal (bet-guess bet) result) collect bet)))
    (mapcar (lambda (winner)
              (let ((x (copy-bet winner)))
                (setf (bet-amount x) (* (bet-rate x) sum))
                x)) (pool-rates winners))))

Yes, I said Django 1.2 comes with built-in user “messages” framework. Great, isn’t it? Django really needed a built-in contrib app to solve this problem. This doesn’t mean, however, that Django-Flash will be discontinued or abandoned.

So, if you use Django-Flash in your projects, don’t worry! Nothing will change. Django-Flash is already compatible with Django 1.2, and we’ll keep improving Django-Flash for a number of reasons.

First of all, we care about the people who use our software. We won’t break your stuff if you decide to upgrade to a newer version of Django – or if you decide to keep using the previous version.

Also, freedom of choice is a good thing. You know, there’s not only one true way to solve every problem. People should be free to choose whatever tool they think is right in a given situation.

For example, based on my first impressions, there are a couple of things I didn’t like at all about the new user “messages” framework:

1. It’s verbose even for the simple cases
2. It ties messages to priority numbers, encouraging you to use it like a logging/debugging framework. I mean, a user “messages” framework is not a logging/debugging framework, right?

But that’s just my opinion. I guess this is what makes open source so great!

I ended up with a Nokia 5800 XpressMusic, a relatively cheap smartphone with a nice feature set. Specifically, what took me for granted was the ability to run Python applications in it!

But what exactly can we do with Python for S60? Well, to answer that question, I decided to implement a non-trivial app to see exactly what it has to offer. The app in question is a Bluetooth remote control called Pytriloquist. (see also screenshots and a video demonstration.)

What’s Good

Platform Independence. The S60 SDK is for Windows only (what a surprise). This is obviously bad for those who use other operating systems. Fortunately, you don’t need a massive SDK to develop and deploy S60 apps if you write them in Python; all you need is Python for S60, a text editor, and a device for testing.

Language and Standard Library. Python is a terrific language: compact, beautiful, powerful, and have an amazing standard library. The good news is that the most popular modules work great on S60. For example, it’s even possible to localize your apps using the gettext module!

What’s Bad

Work in Progress. Python for S60, as long as I know, is a work in progress. This means the language itself works fine, but you can expect missing modules or some bugs in standard library. For example, when writing the Bluetooth client code for my app, I encountered a very annoying bug in which you can only call btsocket.bt_discover() once!

Multi-Module Code. If you are testing your app with Python ScriptShell, everything will work out-of-the-box as long as keep your whole code inside a single massive module. If you break up your code into several modules (or worse, packages) you won’t be able to have modules importing each other unless you adjust the PYTHON_PATH accordingly.

Incomplete and Outdated Documentation. There are some PDFs and API documentation pages available, but they are incomplete and outdated, and this is a bad thing when it comes to dynamic languages; we don’t have a compiler to tell us what’s wrong with our code. This is why Python programmers are so careful about documentation in the first place.

What’s Ugly

Limited Access to Native Features. Specially regarding the native UI system: you cannot change the label of the two standard soft keys, nor add other soft keys, nor add more than two tabs in a tabbed UI, nor have an empty Listbox, nor lots of other things. You can write simple UIs without much effort, that’s true, but those shortcomings just make Python for S60 unusable for more complex projects.

Project Inactivity. The project appears to be on hold. New bug reports are submitted every now and then, but no bugs are closed for six months.

Conclusion

Let’s hope things change for the better after the last announcement made by the Symbian Foundation, but in the meantime, if you pretend to create serious apps for S60, do it in C++.

This is a fairly common thing, at least for the companies I worked for so far, and that’s what scares me the most. Hopefully, you can easily make things better if you want to.

In this post, I’ll show you a few techniques that can be used to write clean JSPs without resorting to any kind of templating system whatsoever.

An example

The first step is to define the base template for the application pages. Consider the following XHTML example and its graphic representation:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta http-equiv="content-type" content="text/html; charset=UTF-8"/>
        <meta name="keywords" content="keyword1, keyword2, keyword3" />
        <meta name="description" content="Page description" />
        <title>Page title</title>
 
        <!-- Essential scripts -->
        <link rel="stylesheet" type="text/css" href="css/style.css"/>
        <script type="text/javascript" src="js/query.js"></script>
 
        <!-- Extra header -->
        <script type="text/javascript" src="js/util.js"></script>
    </head>
    <body>
        <!-- Main -->
        <div id="wrapper">
            <div id="header"></div>
            <div id="content"></div>
            <div id="footer"></div>
        </div> <!-- Main -->
 
        <script type="text/javascript">
            // Other scripts (e.g. Google Analytics)
        </script>
    </body>
</html>

Template graphic representation

What usually happens, even with a simple template such as this, is a high level of code duplication among the different pages due to the heavy use of Copy and Paste Programming:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta http-equiv="content-type" content="text/html; charset=UTF-8"/>
        <meta name="keywords"    content="keyword1, keyword2, keyword3" />
        <meta name="description" content="Page description" />
        <title>Page title</title>
 
        <!-- Essential scripts -->
        <jsp:include page="scripts.jsp"/>
 
        <!-- Extra header -->
        <script type="text/javascript" src="js/util.js"></script>
    </head>
    <body>
        <!-- Main -->
        <div id="wrapper">
            <jsp:include file="header.jsp"/>
            <div id="content">Page content here</div>
            <jsp:include page="footer.jsp"/>
        </div> <!-- Main -->
 
        <script type="text/javascript">
            // Other scripts
        </script>
    </body>
</html>

See? Why break the template into small pieces if you are going keep duplicating code anyway? The best thing to do is to translate that template into something like this:

<%@ taglib tagdir="/WEB-INF/tags/layout" prefix="layout" %>
<layout:page title="Title" description="Page description" keywords="keyword1, keyword2, keyword3">
    <jsp:attribute name="extraHeader">
        <script type="text/javascript" src="js/util.js"></script>
    </jsp:attribute>
    <jsp:attribute name="extraBottom">
        // Other scripts (e.g. Google Analytics)
    </jsp:attribute>
    <jsp:body>
        Page content here
    </jsp:body>
</layout:page>

Content reuse with tag files

There are three types of JSP tags, each with its own strengths and weaknesses: classic tag handlers, simple tag handlers and tag files. Today I’ll cover the basics of tag files, but every Java developer should learn when and how to use the other two.

The first thing we need to do is create the WEB-INF/tags/layout directory, which will keep the tag files we are about to create. Now, create two files inside this directory, one for the header and the other for the footer:

header.tag

<%@ tag body-content="empty" description="Header tag file" %>
<div id="header">
    <!-- Header section here -->
</div>

footer.tag

<%@ tag body-content="empty" description="Footer tag file" %>
<div id="footer">
    <!-- Footer section here -->
</div>

As we can see, tag files are just like JSPs, but with some special properties.

The last tag file, page.tag, defines the overall page structure:

<%@ tag description="Page layout" %>
 
<%@ attribute name="title"       required="true" description="Page title" %>
<%@ attribute name="keywords"    required="true" description="Page keywords to improve SEO" %>
<%@ attribute name="description" required="true" description="Page description" %>
 
<%@ attribute name="extraHeader" fragment="true" description="Extra code to put before </head>" %>
<%@ attribute name="extraBottom" fragment="true" description="Extra code to put before </body>" %>
 
<%@ taglib tagdir="/WEB-INF/tags/layout" prefix="layout" %>
 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta http-equiv="content-type" content="text/html; charset=UTF-8"/>
 
        <!-- Displaying the attributes using EL -->
        <meta name="keywords"    content="${keywords}" />
        <meta name="description" content="${description}" />
        <title>${title}</title>
 
        <!-- Essential scripts -->
        <link rel="stylesheet" type="text/css" href="css/style.css"/>
        <script type="text/javascript" src="js/query.js"></script>
 
        <!-- Process the given input fragment -->
        <jsp:invoke fragment="extraHeader"/>
    </head>
    <body>
        <!-- Main -->
        <div id="wrapper">
            <!-- Renders the page header -->
            <layout:header/>
 
            <!-- Renders the tag body inside a DIV -->
            <div id="content"><jsp:doBody/></div>
 
            <!-- Renders the page footer -->
            <layout:footer/>
        </div> <!-- Main -->
 
        <!-- Process the given input fragment -->
        <jsp:invoke fragment="extraBottom"/>
    </body>
</html>

This last tag file is more complex than the other two, because you can pass to it both values and fragments. Besides, this tag accepts a body, which gets executed with <jsp:doBody/>.

Of course, the example shown here is fairly simple, so we didn’t have to use more advanced features like variables and dynamic attributes. Be that as it may, it’s a good idea to know what the technology has to offer. You’ll never know when a feature will be useful until you need it.

Usage

Now, let’s put those tag files to use:

<%@ taglib tagdir="/WEB-INF/tags/layout" prefix="layout" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<layout:page title="Index page" description="This is the index page of my amazing app" keywords="amazing, app">
    <jsp:attribute name="extraHeader">
        <script type="text/javascript" src="js/util.js"></script>
        <script type="text/javascript" src="js/some_jquery_plugin.js"></script>
    </jsp:attribute>
    <jsp:attribute name="extraBottom">
        $(function() {
            // Call some JQuery function here, specific for this page
        });
    </jsp:attribute>
    <jsp:body>
        Welcome to my amazing app!<br/>
        1+1 = ${1+1}
 
        <ul>
        <c:forEach items="${objects}" var="obj">
            <li>${obj.attr}</li>
        </c:forEach>
        </ul>
    </jsp:body>
</layout:page>

Although the application as a whole follow the same visual standard, each page may need additional resources (JavaScript, CSS, etc.). This is hardly a problem because our template allows us to include extra content if necessary.

The other advantage is that tag files allow the use of custom tags and unified EL, but deny the use of scriptlets, which is a good thing if you want to get rid of messy JSPs.

Recommended reading

• The Java EE 5 Tutorial;
• Head First Servlets & JSP;

A lot of things happened during those 48 hous. The Ticketmaster staff had trouble to keep the website on-line, and due to the enormous load, it’s possible that only a few thousands of people had enough luck to buy their tickets via the website.

Also, Ticketmaster admits that the notification e-mails may not reach, which means we cannot rely on those to follow our orders. In fact, they recomend that customers check their orders manually once a day. Oh boy…

I hate to do certain tasks manually, specially the ones that can be automated. So, in this post I’ll show you a very simple Python-based command-line tool that displays the status of your Ticketmaster¹ orders on the screen², using a technique known as web scraping.

The code, which is also on Github, can be seen below:

#!/usr/bin/env python
#-*- coding:utf-8 -*-
 
import urllib
import urllib2
import pynotify
 
 
_BASE_URL  = 'https://ticketing.ticketmaster.com.br'
_LOGIN_URI = '%s/shwLogin.cfm' % _BASE_URL
_ORDER_URI = '%s/shwCompraDetalhe.cfm?pedidoID=%%s' % _BASE_URL
 
_ORDERS_LINK = '<a href="%s">More...</a>' % _LOGIN_URI
 
_PRIORITY_LOW      = pynotify.URGENCY_LOW
_PRIORITY_NORMAL   = pynotify.URGENCY_NORMAL
_PRIORITY_CRITICAL = pynotify.URGENCY_CRITICAL
 
_NOTIFICATION_TITLE = 'Ticketmaster Order %s'
_STATUS_CODES = {
    'Livre'      : ('Not processed yet'      , _PRIORITY_LOW),
    'StdReserva' : ('Reserving your tickets' , _PRIORITY_NORMAL),
    'StdCobranca': ('Charging your tickets'  , _PRIORITY_NORMAL),
    'VendaOk'    : ('Billed'                 , _PRIORITY_CRITICAL),
    'Recusada'   : ('Rejected'               , _PRIORITY_CRITICAL)
}
 
 
def initialize_notification_system(app_title):
    if not pynotify.init(app_title):
        import sys
        sys.exit(1)
 
 
def display_message(title, message, priority=_PRIORITY_CRITICAL):
    message = pynotify.Notification(title, message)
    message.set_urgency(priority)
    message.show()
 
 
def check_orders(email, password, order_ids):
    try:
        opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
        login(opener, email, password)
        try:
            for order_id in order_ids:
                show_order_status(opener, order_id)
        except:
            title = _NOTIFICATION_TITLE % order_id
            display_message(title, 'Cannot check status')
    except:
        display_message('Ticketmaster', 'Login failed')
 
 
def login(opener, email, password):
    params = urllib.urlencode({'email': email, 'senha': password})
    response = opener.open('%s?tentaLogin=1' % _LOGIN_URI, params)
 
    # Invalid credentials check
    if response.read().find('senha incorreta') >= 0:
        raise Exception
 
 
def show_order_status(opener, order_id):
    response = opener.open(_ORDER_URI % order_id)
    content = response.read()
 
    # Search for the order status codes
    for key, value in _STATUS_CODES.items():
        if content.find(key) >= 0:
            title = _NOTIFICATION_TITLE % order_id
            message = '%s. %s' % (value[0], _ORDERS_LINK)
            display_message(title, message, value[1])
            break
    else:
        display_message(_NOTIFICATION_TITLE % order_id, 'Status not found')
 
 
def main():
    initialize_notification_system("Ticketmaster Order Status")
 
    import optparse
    p = optparse.OptionParser()
 
    # Required command line options
    p.add_option('--email'    , '-e',
        help='E-mail address used in your Ticketmaster credentials.')
    p.add_option('--password' , '-p',
        help='Password used in your Ticketmaster credentials.')
    p.add_option('--order-ids', '-o',
        help='Order IDs to be checked, separated by collons.')
 
    options = p.parse_args()[0]
    if not options.email or not options.password or not options.order_ids:
        p.print_help()
    else:
        order_ids = map(lambda s: s.strip(), options.order_ids.split(','))
        check_orders(options.email, options.password, order_ids)
 
 
if __name__ == '__main__':
    main()

The reason I’ve used web scrapping is because the brazilian Ticketmaster doesn’t seem to provide an API the way Flickr does, for example. Nevertheless, the code is quite simple.

As you can see, it first fires a HTTP request to the login URI in order to validate the user credentials. If the authentication succeeds, our cookie-aware opener will get the session cookies from the server and use them on the following requests, which extract the current order status for each order ID.

Now, let’s give it a spin:

Current order status

Finally, add a new entry to your crontab to run the script whenever you want (e.g. every 10 minutes):

*/10 * * * * python ticketmaster_order_status.py --email=your@email.com --password=passwd --order-ids=1234

Good luck and let there be rock!