Saturday, August 1, 2009

Better Python logging

This is a response to

Back when Python's logging module was being designed, I asserted that there is a much better way to enable and disable logs than a linear sequence of logging levels. That proposal went nowhere, perhaps because people heard it as dismissing the whole notion of having a standard logging module.

In fact, though, it would be easy to graft a small extension on to Python's logging module that would make logging much more flexible. Here's how

1. Add another logging level, say 'custom'. When custom logging is in effect, log messages are generated if they match a **tracepoint name**. Such names are simply strings.

2. Add the following method to the logging module::

logging.custom("abc",'This a custom message logged when "abc" is enabled.')

3. Enable or disable custom tracepoint names with::



Each patten is a string, possibly containing wildcard characters '*' and '?', that matches zero or more tracepoint names. The enable/disable_custom methods simply add or delete entries in a **pattern database** of enable and disabled names.

When logging.custom(s,message) is executed, it searches the pattern database for a match with the string s. It prints the message to the log if a match is indeed found.

That's about it. It's easy to do, compatible with the present logging system, but much more flexible.

Many small tweaks to this scheme are possible, but I'll save those in case there is some real interest.


  1. See also py.log. It is called "keyword-based logging" there.

  2. Edward, perhaps you did not read the relevant section in the documentation at

    which states that:

    "Filters can be used by Handlers and Loggers for more sophisticated filtering than is provided by levels."

    Although the base implementation of Filter implements filtering below a specified point in the logger hierarchy, filters are capable of implementing arbitrary filtering logic, including tracepoint names.

    Did you explore the use of Filters to implement tracepoint names, and if you did, where was the Filter abstraction found wanting? Please feel free to update the Wiki page at

    with your thoughts, so that we can have a centralised place for discussion rather than on individual blogs.

  3. >

    Thanks for this link. It looks like filter subclasses can be made to do what I described. It might be good to provide some convenience methods, but that's pretty much a nit.