C. Scott Ananian

JDoctest!

Sat, 07 Feb 2009 08:33:01 GMT

The Python doctest module is really great: it makes it easy to simultaneously write test suites and demonstrate the usage for your modules. Python's interactive interpreter is key to its coolness: it's really easy to load the code you're working on, type some examples at the prompt, and turn the session into documentation and a test case.

I've been dusting off my Square Dance Revolution project, written in Java, and I thought: gee, it would be nice to use doctests here. A bit of inspiration from doctestj and Rhino, and a bit of elbow grease and: voila! JDoctest is born!

JDoctest is a Javadoc plugin which implements doctests by calling out to the Rhino javascript interpreter. Rhino's interactive javascript session makes Java as fun to program in / debug / test as Python is. (Rhino makes it easy to call between Javascript and Java.) Copy and paste those examples into javadoc comments, add a @doc.test tag, and you've got a test / use case example. I've added hooks to google-code-prettify to make the output beautiful, too.

Here's a simple example using JDoctest, and the SDR sources are now filled with more complex examples (for example). (New SDR release soon, I promise.) Enjoy!

xkcd and shut-the-box

Sat, 03 Jan 2009 05:09:46 GMT

An xkcd comic from a while ago mentioned a "knapsack problem," and one reader (dhogarty) wrote some set enumeration code to tackle it.

Before I go further, I should mention that Knuth is the definitive reference here (as in most things), in particular Volume 4 which is now available only in fascicle form. If you haven't been reading the pre-fascicles just as soon as they're posted on the TAOCP web page, you're a sad sorry person who doesn't deserve to be called a computer scientist. (If you're not a computer scientist, read at least fascicle 4's chapter on the history of combinatorics; you can buy a hardcopy from amazon.)

The xkcd problem is called a "knapsack problem", but it's not, really; see Knuth's commentary around Algorithm F of 7.2.1.3 (vol 4 fascicle 3a). We're generating multicombinations of n items taken from the m appetizers (repetition permitted). It's always worthwhile generating your options in strict lexicographic order to get a handle on duplicates; consult the history of combinatorics cited above for cautionary tales.

A compact way to generate multicombinations without duplicates is simply to require the output to be in lexicographic (sorted) order. For example, tweaking dhogarty's code only slightly:

def all_seqs(size, alphabet):
   for n,c in zip(xrange(len(alphabet)), alphabet):
     for s in all_seqs(size-1, alphabet[n:]) if size>1 else [[]]:
       yield [c]+s

You can wrap this in an iteration of sizes from 0 to infinity to generate multicombinations of all possible cardinalities.

A more elegant solution only generates multicombinations which sum properly (this is similar to Knuth's Algorithm F):

def ways_to_sum(total, menu):
    if total < 0: return
    if total == 0: yield []
    for (item,price),n in zip(menu, xrange(len(menu))):
        for tail in ways_to_sum(total-price, menu[n:]):
            yield [ item ] + tail

menu = [('Fruit', 215), ('Fries', 275),
        ('Salad', 335), ('Wings', 355),
        ('Sticks', 420), ('Sampler', 580)]

for solution in ways_to_sum(1505, menu):
    print solution

It's regrettable that the solution isn't unique. Bad Randall.

I've found lots of opportunities to generate "all possible solutions" in various contexts, and Python's generators have generally been an elegant means to express solutions. For example, I've been discussing "Shut the Box"; here's some code fragments to generate all possible plays for some board state:

def ways_to_sum(total, min_digit=1, max_digit=9):
    """Generate all the ways to sum to the given total using the digits
    `min` through `max` without repetitions.
    By default min and max are 1 and 9, respectively.

    >>> list(ways_to_sum(1))
    [[1]]
    >>> list(ways_to_sum(2))
    [[2]]
    >>> list(ways_to_sum(3))
    [[1, 2], [3]]
    >>> list(ways_to_sum(12))
    [[1, 2, 3, 6], [1, 2, 4, 5], [1, 2, 9], [1, 3, 8], [1, 4, 7], [1, 5, 6], [2, 3, 7], [2, 4, 6], [3, 4, 5], [3, 9], [4, 8], [5, 7]]
    """
    for first in xrange(min_digit, min(max_digit+1, total)):
        for tail in ways_to_sum(total-first, min_digit=first+1, max_digit=max_digit):
            yield [ first ] + tail
    if min_digit <= total and total <= max_digit:
        yield [ total ]

def all_states(min_digit=1):
    """Generate all possible shut-the-box states."""
    if min_digit <= 9:
        for x in all_states(min_digit=min_digit+1):
            yield x
            yield [min_digit] + x
    else:
        yield []

The all_states() function generates the states in lexicographic order based on the set's binary representation (a9 a8 a7 ... a2 a1, to use Knuth's notation).

[This was originally going to be a comment on dhogarty's post, but after writing it I discovered that dhogarty had broken his wordpress installation sometime after 12 Sep 2008. I reverse-engineered the "Captcha-free" spam protection and fixed the Ajax URL in his page, but no dice. Querying the "Captcha-free" implementation with a python one-liner and performing a manual submission still loses. Maybe trackback will work.]

Pippy. And Winnie-the-Pooh.

Tue, 09 Dec 2008 23:24:00 GMT

In October, SJ and I travelled to Peru for OLPC. (Ob. Plug: buy a G1G1 machine this holiday season! It helps get laptops to needy kids and helps fund us to develop more and better software for those kids.)

Anyway, my favorite story from our time in Peru was when SJ stood up in front of a classroom of university kids and announced, in Spanish, "Now we are going to make drawings with pee-pee." He actually was talking about Pippy, the python-programming-for-kids activity written by Chris Ball (with patches from yours truely and others) -- but there's a reason why Pippy is translated "Peppy" in Spanish =).

But I've found another contender for that crown: a cookbook. Best book title ever. (David Friedman not only found that book, he made it an "employee pick" when working at a "large chain bookstore".)

So. Pippy and Winnie-the-Pooh. Closing a scatological circle.

(While I'm in the neighborhood, I have to plug David Friedman's Murder in the Hundred Acre Wood as well. And 10 Lessons from the Movies. Ok, go resume your productive lives now.)

Monkeys!

Tue, 09 Dec 2008 19:25:14 GMT

Dilinger pointed me at an amusing anecdote about monkeys leading off an essay about Python 3.0 (you can skip the Python parts if you're not interested in that stuff). I wonder to what degree some of OLPC's long-running software gun battles are over monkey-and-the-fire-hose sort of stuff. Since I imagine myself on the "good guys" side most of the time (part of having healthy self respect, etc) I like to think of some of the objections I hear to proposed refactorings as being "attacked by the old monkeys" -- but, to be honest, I'm sure there are large parts of Sugar which I like which are just "how it's always been done", and I'm probably as guilty of being an "old monkey" as others.

Binding python to v8

Wed, 05 Nov 2008 17:45:31 GMT

Google's v8 virtual machine (part of Chrome) is really great, from a virtual-machine-implementer's perspective. It would be a fantastic backend for Python, because of the way that object mutation and method dispatch is handled. I thought of hacking together a proof-of-concept, but decided that my work at OLPC wasn't really benefited by my going off to spend a year writing a fast Python runtime.

Luckily, I don't have to: pyv8 is a proof-of-concept implementation of just such a thing! And it's ten times faster than standard interpreted Python -- although it should be noted that this is for a strictly toy benchmark.

What's missing is bypassing pyjamas and working directly from python bytecode, and a better attempt at providing python standard library support. Hopefully other bright minds are hard at work on this!