programming, python September 3, 2010 Jack No comments

Using Decorators for Flexible Prompts

As a general rule, anything that you’re going to need to do many times should be abstracted to be made easy. When you’re coding something with a prompt, adding commands definitely falls into that category. With Python introspection and decorators make a new text prompt can be as simple as writing a function and defining a simple format for it. Like this:

    @command_format(("terms","listof_uint"))
    def sumstar(self, **kwargs):
        print sum(kwargs["terms"])

Which is safe, clear, and working. This method has a number of advantages:

  1. It easily allows for features like automatically prompting for missing arguments and argument defaults.
  2. The decorator for each command function doubles as documentation since it both simply lists the names of the keyword arguments as well as their types.
  3. Cleanly separates the function of the command from the validation of its arguments.
  4. Ideal for allowing users to write pluggable commands because actual command functions use native objects instead of strings, and they automatically receive the benefits of prompting, listing, defaults, etc. Plus, there’s only a single line of overhead when the validators are pre-defined.
  5. Allows validators to easily use each other so multiple minor variations of the same type of validation can be expressed simply.

The command_format Meta-Decorator

def command_format(*types):
    def cf(fn):
        def cfdec(self, **kwargs):

            rem = kwargs["args"]
            realkwargs = {}

            for kw, validator in types:
                validator = getattr(self, validator)
                valid, result, rem = validator(rem.lstrip())
                if not valid:
                    log.debug("Couldn't properly parse %s" % kwargs["args"])
                    return

                realkwargs[kw] = result

            return fn(self, **realkwargs)
        return cfdec
    return cf

It’s not every day that you see a triple nested def, but it’s pretty standard meta-decorator, where *types is a list of string tuples like [("keyword","validator")]. Where each keyword is the name of the keyword argument the parsed object will be passed as to the final command function and each validator is a string-reference to a method of the class this command resides in. If you’re operating outside of a class, this validator could easily be a function reference as well.

This meta-decorator is applied to every function that embodies a command. We’ll get to that later. For now, let’s look at what these validators look like.

A Simple Validator

Validators are passed one argument: the entire unparsed portion of the string you’re parsing, which can be ""/None. It returns a three tuple of objects. A boolean value whether the validator passed and, if it did, the resulting object and the remaining, unparsed portion of the string. Let’s take a look at a simple validator that will take a positive integer.

class MyCommandHandler(...)
    ...
    def uint(self, args):
        terms = args.split()
        if not terms:
            return (False, None, None)

        try:
            ret = int(terms[0])
        except:
            log.error("Couldn't parse %s as integer!" % terms[0])
            return (False, None, None)

        if ret < 0:
            log.error("Argument must be > 0")
            return (False, None, None)

        return (True, ret, " ".join(terms[1:]))

Extremely simple, it grabs the first whitespace delimited argument from the string, attempts to parse it as an integer and makes sure it’s >0 before returning that the arg passes validation.

Let’s put our command_format and the int validator to use by writing a simple sum that will take a string argument, attempt to parse out two integers and print the sum.

sum Example

class MyCommandHandler(...)
    ...
    @command_format(("term1","uint"),("term2","uint"))
    def sum(self, **kwargs):
        print kwargs["term1"] + kwargs["term2"]

No need to do any extra validation on either argument, we know both that they exist and they’re integers, so we’re safe.

At this point, with a little wrapping code we can do the following:

jack@arpeggi:~ $ ./basicsum.py
sum 12 16
28

sum 1 b
Couldn't parse b as integer!
Couldn't properly parse 1 b

But that’s not very exciting. Let’s look a little harder and see how we can leverage these.

More Useful Validators

Automatic Argument Prompting

The first thing that pops to mind as being a useful addition to a command prompt is being a little more tolerant to missing arguments. Let’s make our uint validator prompt instead of bailing on an empty string.

class MyCommandHandler(...)
    ...
    def uint(self, args):
        if not args:
            args = raw_input("uint: ")

        terms = args.split()
        if not terms:
            return (False, None, None)

        try:
            ret = int(terms[0])
        except:
            log.error("Couldn't parse %s as integer!" % terms[0])
            return (False, None, None)

        if ret < 0:
            log.error("Argument must be > 0")
            return (False, None, None)

        return (True, ret, " ".join(terms[1:]))

NOTE: One of the attributes of the command_handler resulting decorator is that the arguments passed to each validator are stripped of leading whitespace so we know that if not args will work on any empty string passed to the validator.

Now, again with a shade of wrapper code, we have a bit more functionality.

jack@arpeggi:~/blog $ ./promptsum.py
sum
uint: 13
uint: 14
27

sum 13
uint: 20
33

sum 14 15
29

Function on Lists of Arbitrary Types

So, at this point, we’ve basically got a sum2 function and, even though it’s nice (=P), it’s pretty inflexible. Let’s write another validator that generates a list of uints and work on that, with another appropriate function. Better yet, let’s write a validator that will make a list of *any* consistent type.

class MyCommandHandler(...)
    ...
    def listof_(self, prompt, val, args):
        l = []
        if not args:
            args = raw_input(prompt)

        while args:
            v, term, args = val(args.strip())
            if v:
                l.append(term)
            else:
                # If you wanted to be really fault tolerant, you could 
                # remove the first term and try again. We'll be really 
                # strict here though.
                return (False, None, None)
        return (True, l, "")

    def listof_uint(self, args):
        return self.listof_("uints: ", self.uint, args)

There we go. As you can see, the listof_ function will attempt to use any given validator on the input until it’s used up. A smarter implementation might not use up the whole input, but expect lists to be bracketed by symbols, or a certain length. This also includes the arbitrary prompt that we used before. Let’s put it to use implementing sum*.

    @command_format(("terms","listof_uint"))
    def sumstar(self, **kwargs):
        print sum(kwargs["terms"])

Done. Now we have a much more flexible sum* command. As evidenced using starsum.py.

jack@arpeggi:~/blog $ ./starsum.py
sum*
uints: 4 7 8
19

sum* 3 8 10 11
32

sum* 4 5 6 a
Couldn't parse a as integer!
Couldn't properly parse  4 5 6 a

Closing Notes + Source

The above examples use positive integers (and yes, I’m aware that *sum* works on negative integers a well =P) because they’re a convenient type that everybody knows. One of the advantages of the system is that it doesn’t matter what sort of return object the validator gives, the command functions can just assume everything is all right because it never gets called if the args don’t parse correctly. I wrote a system very much like this for the new version of my side project Canto (an RSS reader) that allows users to give a list of story indexes, which are then converted from simple integers to actual story objects by the validators (listof_items) and passed to the corresponding function.

Also, this code is easily extensible to use any sort of input format (obviously raw_input isn’t exactly the most useful way to get a string from the user if you’re graphical or running ncurses, like I always am). Canto uses this via a Textbox class from a curses window.

The examples used in this write-up are available as executable .py files (used for the output sections).

Basic sum2 validating prompt: basicsum.py
Added missing argument prompting: promptsum.py
Added sum*: starsum.py

These were tested on Linux with Python 2.6, but should work on practically any platform with any relatively modern Python. It’s also under public domain, if you actually care about the licensing of blog snippets =).

:wq

Leave a Reply

Your email address will not be published. Required fields are marked *