I can't get my head around Python's logging module. My needs are very simple: I just want to log everything to syslog. After reading documentation I came up with this simple test script:
import logging
import logging.handlers
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)
handler = logging.handlers.SysLogHandler()
my_logger.addHandler(handler)
my_logger.debug('this is debug')
my_logger.critical('this is critical')
But this script does not produce any log records in syslog. What's wrong?
Here's the yaml dictConfig way recommended for 3.2 & later.
In log cfg.yml:
version: 1
disable_existing_loggers: true
formatters:
default:
format: "[%(process)d] %(name)s(%(funcName)s:%(lineno)s) - %(levelname)s: %(message)s"
handlers:
syslog:
class: logging.handlers.SysLogHandler
level: DEBUG
formatter: default
address: /dev/log
facility: local0
rotating_file:
class: logging.handlers.RotatingFileHandler
level: DEBUG
formatter: default
filename: rotating.log
maxBytes: 10485760 # 10MB
backupCount: 20
encoding: utf8
root:
level: DEBUG
handlers: [syslog, rotating_file]
propogate: yes
loggers:
main:
level: DEBUG
handlers: [syslog, rotating_file]
propogate: yes
Load the config using:
log_config = yaml.safe_load(open('cfg.yml'))
logging.config.dictConfig(log_config)
Configured both syslog & a direct file. Note that the /dev/log is OS specific.
import syslog
syslog.openlog(ident="LOG_IDENTIFIER",logoption=syslog.LOG_PID, facility=syslog.LOG_LOCAL0)
syslog.syslog('Log processing initiated...')
the above script will log to LOCAL0 facility with our custom "LOG_IDENTIFIER"... you can use LOCAL[0-7] for local purpose.
From https://github.com/luismartingil/per.scripts/tree/master/python_syslog
#!/usr/bin/python
# -*- coding: utf-8 -*-
'''
Implements a new handler for the logging module which uses the pure syslog python module.
@author: Luis Martin Gil
@year: 2013
'''
import logging
import syslog
class SysLogLibHandler(logging.Handler):
"""A logging handler that emits messages to syslog.syslog."""
FACILITY = [syslog.LOG_LOCAL0,
syslog.LOG_LOCAL1,
syslog.LOG_LOCAL2,
syslog.LOG_LOCAL3,
syslog.LOG_LOCAL4,
syslog.LOG_LOCAL5,
syslog.LOG_LOCAL6,
syslog.LOG_LOCAL7]
def __init__(self, n):
""" Pre. (0 <= n <= 7) """
try:
syslog.openlog(logoption=syslog.LOG_PID, facility=self.FACILITY[n])
except Exception , err:
try:
syslog.openlog(syslog.LOG_PID, self.FACILITY[n])
except Exception, err:
try:
syslog.openlog('my_ident', syslog.LOG_PID, self.FACILITY[n])
except:
raise
# We got it
logging.Handler.__init__(self)
def emit(self, record):
syslog.syslog(self.format(record))
if __name__ == '__main__':
""" Lets play with the log class. """
# Some variables we need
_id = 'myproj_v2.0'
logStr = 'debug'
logFacilityLocalN = 1
# Defines a logging level and logging format based on a given string key.
LOG_ATTR = {'debug': (logging.DEBUG,
_id + ' %(levelname)-9s %(name)-15s %(threadName)-14s +%(lineno)-4d %(message)s'),
'info': (logging.INFO,
_id + ' %(levelname)-9s %(message)s'),
'warning': (logging.WARNING,
_id + ' %(levelname)-9s %(message)s'),
'error': (logging.ERROR,
_id + ' %(levelname)-9s %(message)s'),
'critical': (logging.CRITICAL,
_id + ' %(levelname)-9s %(message)s')}
loglevel, logformat = LOG_ATTR[logStr]
# Configuring the logger
logger = logging.getLogger()
logger.setLevel(loglevel)
# Clearing previous logs
logger.handlers = []
# Setting formaters and adding handlers.
formatter = logging.Formatter(logformat)
handlers = []
handlers.append(SysLogLibHandler(logFacilityLocalN))
for h in handlers:
h.setFormatter(formatter)
logger.addHandler(h)
# Yep!
logging.debug('test debug')
logging.info('test info')
logging.warning('test warning')
logging.error('test error')
logging.critical('test critical')
Is your syslog.conf set up to handle facility=user?
You can set the facility used by the python logger with the facility argument, something like this:
handler = logging.handlers.SysLogHandler(facility=SysLogHandler.LOG_DAEMON)
You should always use the local host for logging, whether to /dev/log or localhost through the TCP stack. This allows the fully RFC compliant and featureful system logging daemon to handle syslog. This eliminates the need for the remote daemon to be functional and provides the enhanced capabilities of syslog daemon's such as rsyslog and syslog-ng for instance. The same philosophy goes for SMTP. Just hand it to the local SMTP software. In this case use 'program mode' not the daemon, but it's the same idea. Let the more capable software handle it. Retrying, queuing, local spooling, using TCP instead of UDP for syslog and so forth become possible. You can also [re-]configure those daemons separately from your code as it should be.
Save your coding for your application, let other software do it's job in concert.
Piecing things together from here and other places, this is what I came up with that works on unbuntu 12.04 and centOS6
Create an file in /etc/rsyslog.d/ that ends in .conf and add the following text
local6.* /var/log/my-logfile
Restart rsyslog, reloading did NOT seem to work for the new log files. Maybe it only reloads existing conf files?
sudo restart rsyslog
Then you can use this test program to make sure it actually works.
import logging, sys
from logging import config
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': '%(levelname)s %(module)s P%(process)d T%(thread)d %(message)s'
},
},
'handlers': {
'stdout': {
'class': 'logging.StreamHandler',
'stream': sys.stdout,
'formatter': 'verbose',
},
'sys-logger6': {
'class': 'logging.handlers.SysLogHandler',
'address': '/dev/log',
'facility': "local6",
'formatter': 'verbose',
},
},
'loggers': {
'my-logger': {
'handlers': ['sys-logger6','stdout'],
'level': logging.DEBUG,
'propagate': True,
},
}
}
config.dictConfig(LOGGING)
logger = logging.getLogger("my-logger")
logger.debug("Debug")
logger.info("Info")
logger.warn("Warn")
logger.error("Error")
logger.critical("Critical")
I found the syslog module to make it quite easy to get the basic logging behavior you describe:
import syslog
syslog.syslog("This is a test message")
syslog.syslog(syslog.LOG_INFO, "Test message at INFO priority")
There are other things you could do, too, but even just the first two lines of that will get you what you've asked for as I understand it.
You can also add a file handler or rotating file handler to send your logs to a local file: http://docs.python.org/2/library/logging.handlers.html
I add a little extra comment just in case it helps anyone because I found this exchange useful but needed this little extra bit of info to get it all working.
To log to a specific facility using SysLogHandler you need to specify the facility value. Say for example that you have defined:
local3.* /var/log/mylog
in syslog, then you'll want to use:
handler = logging.handlers.SysLogHandler(address = ('localhost',514), facility=19)
and you also need to have syslog listening on UDP to use localhost instead of /dev/log.
I fix it on my notebook. The rsyslog service did not listen on socket service.
I config this line bellow in /etc/rsyslog.conf file and solved the problem:
$SystemLogSocketName /dev/log
Source: Stackoverflow.com