| Andrew Cooke | Contents | Latest | RSS | Twitter | Previous | Next

C[omp]ute

Welcome to my blog, which was once a mailing list of the same name and is still generated by mail. Please reply via the "comment" links.

Always interested in offers/projects/new ideas. Eclectic experience in fields like: numerical computing; Python web; Java enterprise; functional languages; GPGPU; SQL databases; etc. Based in Santiago, Chile; telecommute worldwide. CV; email.

Personal Projects

Lepl parser for Python.

Colorless Green.

Photography around Santiago.

SVG experiment.

Professional Portfolio

Calibration of seismometers.

Data access via web services.

Cache rewrite.

Extending OpenSSH.

C-ORM: docs, API.

Last 100 entries

And Smugness; McCloskey Economics Trilogy; cmocka - Mocks for C; Concept Creep (Americans); Futhark - OpenCL Language; Moved / Gone; Fan and USB issues; Burgers in Santiago; The Origin of Icosahedral Symmetry in Viruses; autoenum on PyPI; Jars Explains; Tomato Chutney v3; REST; US Elections and Gender: 24 Point Swing; PPPoE on OpenSuse Leap 42.1; SuperMicro X10SDV-TLN4F/F with Opensuse Leap 42.1; Big Data AI Could Be Very Bad Indeed....; Cornering; Postcapitalism (Paul Mason); Black Science Fiction; Git is not a CDN; Mining of Massive Data Sets; Rachel Kaadzi Ghansah; How great republics meet their end; Raspberry, Strawberry and Banana Jam; Interesting Dead Areas of Math; Later Taste; For Sale; Death By Bean; It's Good!; Tomato Chutney v2; Time ATAC MX 2 Pedals - First Impressions; Online Chilean Crafts; Intellectual Variety; Taste + Texture; Time Invariance and Gauge Symmetry; Jodorowsky; Tomato Chutney; Analysis of Support for Trump; Indian SF; TP-Link TL-WR841N DNS TCP Bug; TP-Link TL-WR841N as Wireless Bridge; Sending Email On Time; Maybe run a command; Sterile Neutrinos; Strawberry and Banana Jam; The Best Of All Possible Worlds; Kenzaburo Oe: The Changeling; Peach Jam; Taste Test; Strawberry and Raspberry Jam; flac to mp3 on OpenSuse 42.1; Also, Sebald; Kenzaburo Oe Interview; Otake (Kitani Minoru) move Black 121; Is free speech in British universities under threat?; I am actually good at computers; Was This Mansplaining?; WebFaction / LetsEncrypt / General Disappointment; Sensible Philosophy of Science; George Ellis; Misplaced Intuition and Online Communities; More Reading About Japan; Visibilty / Public Comments / Domestic Violence; Ferias de Santiago; More (Clearly Deliberate); Deleted Obit Post; And then a 50 yo male posts this...; We Have Both Kinds Of Contributors; Free Springer Books; Books on Religion; Books on Linguistics; Palestinan Electronica; Books In Anthropology; Taylor Expansions of Spacetime; Info on Juniper; Efficient Stream Processing; The Moral Character of Crypto; Hearing Aid Info; Small Success With Go!; Re: Quick message - This link is broken; Adding Reverb To The Echo Chamber; Sox Audio Tools; Would This Have Been OK?; Honesty only important economically before institutions develop; Stegangraphy via PS4; OpenCL Mess; More Book Recommendations; Good Explanation of Difference Between Majority + Minority; Musical Chairs - Who's The Privileged White Guy; I can see straight men watching this conversation and laffing; Meta Thread Defending POC Causes POC To Close Account; Indigenous People Of Chile; Curry Recipe; Interesting Link On Marginality; A Nuclear Launch Ordered, 1962; More Book Recs (Better Person); It's Nuanced, And I Tried, So Back Off; Marx; The Negative Of Positive; Jenny Holzer Rocks

© 2006-2015 Andrew Cooke (site) / post authors (content).

Logging in Python

From: "andrew cooke" <andrew@...>

Date: Sat, 26 Apr 2008 10:49:29 -0400 (CLT)

There's hardly any docs about this (apart from the way over-detailed
no-wood-for-the-trees library docs), but it turns out Python does have a
good (but broken!) logging system.

Motivated by my experience with Java logging I want:
- An external config that lets me control the level and location of messages
- Logging that is identified with modules, for easy filtering
  (module level is simpler than class level - see below - except that
  it tickles a bug, apparently)

And this is how I do it:

When the app starts, it calls

import logging
import logging.config
import sys
from acooke.util.path import sys_path_find
def init_log(name):
    try:
        path = sys_path_find(name)
        logging.config.fileConfig(path)
        log = logging.getLogger(__name__)
        log.info('Configured logging from %s' % path)
    except:
        logging.exception('Failed to start logging: %s: %s'
                          % sys.exc_info()[:2])
        logging.warning('Will use default logging configuration')
        logging.warning('To fix this, put configuration in %s on $PYTHONPATH'
                        % name)

where sys_path_find runs through the system path looking for the named
file.  This loads the config, which is then accessed within modules as:

from acooke.util.log import Log
log = Log(__name__)

class Foo(object):
    def foo(self):
        log.debug("My debug message")

Which generates a log message associated with the module name.  You would
expect this to look like:

import logging
log = logging.getLoger(__name__)

but that doesn't work.  Instead, the Log class above lazily resolves the
logger instance so that creation happens *after* the configuration
described above has occurred.

class Log(object):
    def __init__(self, name):
        super(Log, self).__init__()
        self._name = name
        self._lazy = None
    def __getattr__(self, key):
        if not self._lazy:
            self._lazy = logging.getLogger(self._name)
        return getattr(self._lazy, key)

And finally, here's a typical config.  I send everything to the same
handler (syserr) with the same format, but i define three different
loggers.  One handles everything from modules in the "dbmac.*" tree (the
application), another handles modules in the "acooke.*" tree (libraries)
and a thrird is a catch all for anything else ("root" is a reserved name).


[loggers]
keys=root,dbmac,acooke

[handlers]
keys=common

[formatters]
keys=common

[logger_root]
# configure level for system logs (if any) here
level=NOTSET
handlers=common

[logger_dbmac]
# configure level for main application here
level=DEBUG
propagate=0
qualname=dbmac
handlers=common

[logger_acooke]
# configure level for acooke libraries here
level=DEBUG
propagate=0
qualname=acooke
handlers=common

[handler_common]
level=NOTSET
class=StreamHandler
args=(sys.stderr,)
formatter=common

[formatter_common]
class=logging.Formatter
datefmt=
format=%(asctime)s [%(levelname)9s %(name)-30s] %(message)s

Typical log output looks like:

2008-04-26 08:54:44,209 [     INFO acooke.util.log               ]
Configured logging from
/home/andrew/projects/personal/src/python/dbmac/dbmac/src/dbmac.ini
2008-04-26 08:54:49,224 [    DEBUG acooke.wx.validators          ] Engine
:= 789
2008-04-26 08:54:49,224 [    DEBUG acooke.wx.validators          ] Name :=
789

Andrew

Comment on this post