Python Rocks! and other rants

Weblog of Kent S Johnson

#70 2005-12-28 10:49:20

Why I love Python 5

// The XPathAPI is in different packages in Java 1.4 and 1.5.
// Use introspection to find the right one
private static Method __selectSingleNode;

static {
    // Look for the XPathAPI class in two places
    Class XPathAPI = null;
    try {
        XPathAPI = Class.forName("org.apache.xpath.XPathAPI");
    } catch (ClassNotFoundException e) {
        try {
            XPathAPI = Class.forName("com.sun.org.apache.xpath.internal.XPathAPI");
        } catch (ClassNotFoundException e1) {
        }
    }

    // Get the methods we support
    try {
        __selectSingleNode =
          XPathAPI.getMethod("selectSingleNode",
                             new Class[] { Node.class, String.class} );
    } catch (SecurityException e) {
    } catch (NoSuchMethodException e) {
    }
}

/** XPathAPI.selectSingleNode */
public static Node selectSingleNode(Node node, String xpath) {
    try {
        return (Node)__selectSingleNode.invoke(null, new Object[] { node, xpath });
    } catch (IllegalArgumentException e) {
        e.printStackTrace();
    } catch (IllegalAccessException e) {
        e.printStackTrace();
    } catch (InvocationTargetException e) {
        e.printStackTrace();
    }
    return null;
}

try:
  import org.apache.xpath.XPathAPI as XPathAPI
except ImportError:
  import com.sun.org.apache.xpath.internal.XPathAPI as XPathAPI

__selectSingleNode = getattr(XPathAPI, 'selectSingleNode')

try:
  __selectSingleNode = ...
except (SecurityException, NoSuchMethodException), e:
  e.printStackTrace()

Categories: Java, Python

#69 2005-12-06 22:37:36

Simple itertools.groupby() example

Suppose you have a (sorted) list of dicts containing the names of cities and states, and you want to print them out with headings by state:

>>> cities = [
...     { 'city' : 'Harford', 'state' : 'Connecticut' },
...     { 'city' : 'Boston', 'state' : 'Massachusetts' },
...     { 'city' : 'Worcester', 'state' : 'Massachusetts' },
...     { 'city' : 'Albany', 'state' : 'New York' },
...     { 'city' : 'New York City', 'state' : 'New York' },
...     { 'city' : 'Yonkers', 'state' : 'New York' },
... ]

First let me explain operator.itemgetter(). This function is a factory for new functions. It creates functions that access items using a key. In this case I will use it to create a function to access the 'state' item of each record:

>>> from operator import itemgetter
>>> getState = itemgetter('state')
>>> getState
<operator.itemgetter object at 0x00A31D90>
>>> getState(cities[0])
'Connecticut'
>>> [ getState(record) for record in cities ]
['Connecticut', 'Massachusetts', 'Massachusetts', 'New York', 'New York', 'New York']

So the value returned by itemgetter('state') is a function that accepts a dict as an argument and returns the 'state' item of the dict. Calling getState(d) is the same as writing d['state'].

What does this have to do with itertool.groupby()?

>>> from itertools import groupby
>>> help(groupby)
Help on class groupby in module itertools:

class groupby(__builtin__.object)
|  groupby(iterable[, keyfunc]) -> create an iterator which returns
|  (key, sub-iterator) grouped by each value of key(value).

groupby() takes an optional second argument which is a function to extract keys from the data. getState() is just the function we need.

>>> groups = groupby(cities, getState)
>>> groups
<itertools.groupby object at 0x00A88300>

Hmm. That's a bit opaque. groupby() returns an iterator. Each item in the iterator is a pair of (key, group). Let's take a look:

>>> for key, group in groups:
...   print key, group
...
Connecticut <itertools._grouper object at 0x0089D0F0>
Massachusetts <itertools._grouper object at 0x0089D0C0>
New York <itertools._grouper object at 0x0089D0F0>

Hmm. Still a bit opaque :-) The key part is clear - that's the state, extracted with getState - but group is another iterator. One way to look at it's contents is to use a nested loop. Note that I have to call groupby() again, the old iterator was consumed by the last loop:

>>> for key, group in groupby(cities, getState):
...   print key
...   for record in group:
...     print record
...
Connecticut
{'city': 'Harford', 'state': 'Connecticut'}
Massachusetts
{'city': 'Boston', 'state': 'Massachusetts'}
{'city': 'Worcester', 'state': 'Massachusetts'}
New York
{'city': 'Albany', 'state': 'New York'}
{'city': 'New York City', 'state': 'New York'}
{'city': 'Yonkers', 'state': 'New York'}

Well, that makes more sense! And it's not too far from the original requirement, we just need to pretty up the output a bit. How about this:

>>> for key, group in groupby(cities, getState):
...   print 'State:', key
...   for record in group:
...     print '   ', record['city']
...
State: Connecticut
     Harford
State: Massachusetts
     Boston
     Worcester
State: New York
     Albany
     New York City
     Yonkers

Other than misspelling Hartford (sheesh, and I grew up in Connecticut!) that's not too bad!

Categories: Python

#68 2005-12-03 07:01:04

How I write code

I tend to design from the bottom up - not exclusively, but in general I make small parts and combine them to make larger parts until I have something that does what I want. I refactor constantly as my understanding of a problem and the solution increase. This way I always have complete working code for some section of the problem. I rarely use stubs of any kind.

To start I will take some small section of the problem and think about what kind of data and operations on the data I need to solve it. For a very simple problem I might just write some functions to operate on the data. As I expand into larger parts of the problem I might find that several functions are operating on the same data and decide that they belong in a class. Or it might be clear from the start that I want to create a class around the data.

When one chunk is done to my satisfaction, I take on another, and another. I am creating building blocks, then using the building blocks to create larger blocks. Some of the blocks are classes, others are functions.

I write unit tests as I go, sometimes test-first, sometimes test-after, but always alternating writing code with writing tests so I know the code works and I have a safety net when I need to refactor or make other major changes.

At any time I may discover that I made a bad decision earlier, or realize that there is a better way to structure the code or data. Then I stop and rework until I am happy with what I have. The unit tests give me confidence that I haven's broken anything in the process. It's a very organic process, I sometimes think of it as growing a program.

(from a post to the Python-tutor list)

Categories: Agile