| Andrew Cooke | Contents | Latest | RSS | Twitter | Previous | Next

C[omp]ute

Welcome to my blog, which was once a mailing list of the same name and is still generated by mail. Please reply via the "comment" links.

Always interested in offers/projects/new ideas. Eclectic experience in fields like: numerical computing; Python web; Java enterprise; functional languages; GPGPU; SQL databases; etc. Based in Santiago, Chile; telecommute worldwide. CV; email.

Personal Projects

Lepl parser for Python.

Colorless Green.

Photography around Santiago.

SVG experiment.

Professional Portfolio

Calibration of seismometers.

Data access via web services.

Cache rewrite.

Extending OpenSSH.

Last 100 entries

I Want To Be A Redneck!; Reverse Racism; The Lost Art Of Nomography; IBM Data Center (Photo); Interesting Account Of Gamma Hack; The Most Interesting Audiophile In The World; How did the first world war actually end?; Ky - Restaurant Santiago; The Black Dork Lives!; The UN Requires Unaninmous Decisions; LPIR - Steganography in Practice; How I Am 6; Clear Explanation of Verizon / Level 3 / Netflix; Teenage Girls; Formalising NSA Attacks; Switching Brakes (Tektro Hydraulic); Naim NAP 100 (Power Amp); AKG 550 First Impressions; Facebook manipulates emotions (no really); Map Reduce "No Longer Used" At Google; Removing RAID metadata; New Bike (Good Bike Shop, Santiago Chile); Removing APE Tags in Linux; Compiling Python 3.0 With GCC 4.8; Maven is Amazing; Generating Docs from a GitHub Wiki; Modular Shelves; Bash Best Practices; Good Emergency Gasfiter (Santiago, Chile); Readings in Recent Architecture; Roger Casement; Integrated Information Theory (Or Not); Possibly undefined macro AC_ENABLE_SHARED; Update on Charges; Sunburst Visualisation; Spectral Embeddings (Distances -> Coordinates); Introduction to Causality; Filtering To Help Colour-Blindness; ASUS 1015E-DS02 Too; Ready Player One; Writing Clear, Fast Julia Code; List of LatAm Novels; Running (for women); Building a Jenkins Plugin and a Jar (for Command Line use); Headphone Test Recordings; Causal Consistency; The Quest for Randomness; Chat Wars; Real-life Financial Co Without ACID Database...; Flexible Muscle-Based Locomotion for Bipedal Creatures; SQL Performance Explained; The Little Manual of API Design; Multiple Word Sizes; CRC - Next Steps; FizzBuzz; Update on CRCs; Decent Links / Discussion Community; Automated Reasoning About LLVM Optimizations and Undefined Behavior; A Painless Guide To CRC Error Detection Algorithms; Tests in Julia; Dave Eggers: what's so funny about peace, love and Starship?; Cello - High Level C Programming; autoreconf needs tar; Will Self Goes To Heathrow; Top 5 BioInformatics Papers; Vasovagal Response; Good Food in Vina; Chilean Drug Criminals Use Subsitution Cipher; Adrenaline; Stiglitz on the Impact of Technology; Why Not; How I Am 5; Lenovo X240 OpenSuse 13.1; NSA and GCHQ - Psychological Trolls; Finite Fields in Julia (Defining Your Own Number Type); Julian Assange; Starting Qemu on OpenSuse; Noisy GAs/TMs; Venezuela; Reinstalling GRUB with EFI; Instructions For Disabling KDE Indexing; Evolving Speakers; Changing Salt Size in Simple Crypt 3.0.0; Logarithmic Map (Moved); More Info; Words Found in Voynich Manuscript; An Inventory Of 3D Space-Filling Curves; Foxes Using Magnetic Fields To Hunt; 5 Rounds RC5 No Rotation; JP Morgan and Madoff; Ori - Secure, Distributed File System; Physical Unclonable Functions (PUFs); Prejudice on Reddit; Recursion OK; Optimizing Julia Code; Cash Handouts in Brazil; Couple Nice Music Videos; It Also Works!; Adaptive Plaintext; It Works!; RC5 Without Rotation (2)

© 2006-2013 Andrew Cooke (site) / post authors (content).

Logging in Python

From: "andrew cooke" <andrew@...>

Date: Sat, 26 Apr 2008 10:49:29 -0400 (CLT)

There's hardly any docs about this (apart from the way over-detailed
no-wood-for-the-trees library docs), but it turns out Python does have a
good (but broken!) logging system.

Motivated by my experience with Java logging I want:
- An external config that lets me control the level and location of messages
- Logging that is identified with modules, for easy filtering
  (module level is simpler than class level - see below - except that
  it tickles a bug, apparently)

And this is how I do it:

When the app starts, it calls

import logging
import logging.config
import sys
from acooke.util.path import sys_path_find
def init_log(name):
    try:
        path = sys_path_find(name)
        logging.config.fileConfig(path)
        log = logging.getLogger(__name__)
        log.info('Configured logging from %s' % path)
    except:
        logging.exception('Failed to start logging: %s: %s'
                          % sys.exc_info()[:2])
        logging.warning('Will use default logging configuration')
        logging.warning('To fix this, put configuration in %s on $PYTHONPATH'
                        % name)

where sys_path_find runs through the system path looking for the named
file.  This loads the config, which is then accessed within modules as:

from acooke.util.log import Log
log = Log(__name__)

class Foo(object):
    def foo(self):
        log.debug("My debug message")

Which generates a log message associated with the module name.  You would
expect this to look like:

import logging
log = logging.getLoger(__name__)

but that doesn't work.  Instead, the Log class above lazily resolves the
logger instance so that creation happens *after* the configuration
described above has occurred.

class Log(object):
    def __init__(self, name):
        super(Log, self).__init__()
        self._name = name
        self._lazy = None
    def __getattr__(self, key):
        if not self._lazy:
            self._lazy = logging.getLogger(self._name)
        return getattr(self._lazy, key)

And finally, here's a typical config.  I send everything to the same
handler (syserr) with the same format, but i define three different
loggers.  One handles everything from modules in the "dbmac.*" tree (the
application), another handles modules in the "acooke.*" tree (libraries)
and a thrird is a catch all for anything else ("root" is a reserved name).


[loggers]
keys=root,dbmac,acooke

[handlers]
keys=common

[formatters]
keys=common

[logger_root]
# configure level for system logs (if any) here
level=NOTSET
handlers=common

[logger_dbmac]
# configure level for main application here
level=DEBUG
propagate=0
qualname=dbmac
handlers=common

[logger_acooke]
# configure level for acooke libraries here
level=DEBUG
propagate=0
qualname=acooke
handlers=common

[handler_common]
level=NOTSET
class=StreamHandler
args=(sys.stderr,)
formatter=common

[formatter_common]
class=logging.Formatter
datefmt=
format=%(asctime)s [%(levelname)9s %(name)-30s] %(message)s

Typical log output looks like:

2008-04-26 08:54:44,209 [     INFO acooke.util.log               ]
Configured logging from
/home/andrew/projects/personal/src/python/dbmac/dbmac/src/dbmac.ini
2008-04-26 08:54:49,224 [    DEBUG acooke.wx.validators          ] Engine
:= 789
2008-04-26 08:54:49,224 [    DEBUG acooke.wx.validators          ] Name :=
789

Andrew

Comment on this post