How to log from your Python web app to the Syslog on OpenBSD

I have an OpenBSD VPS that runs a few web apps that I’ve written. The web apps are written in Python and are run by Gunicorn web server.

I wanted a simple logging solution that would use the standard Python logging library and would not require much extra setup on the server. Another requirement I had was to log everything - application logs, error & access logs from Gunicorn, to the same file.

The cornerstone of the solution I settled on is the built-in Syslog facilities of OpenBSD. This way I would avoid the need to do log file rotation from my application, worry about file permissions etc.

The setup was a bit bumpy so I’m documenting it here for my future self…

There are 3 main parts to configure:

1. Python logging to log to the Gunicorn logger
2. Gunicorn to log to the Syslog
3. Syslog to write all output to a dedicated log file

Configure Python to log to the Gunicorn logger

To configure logging library to do this:

import logging


logger = logging.getLogger("gunicorn.error")
logger.info("Hello, world!")

I’m logging to the gunicorn.error logger, the same Gunicorn is using. This way I don’t need any extra configuration to combine logs.

Configure Gunicorn to log to the Syslog

This is what’s needed in Gunicorn configuration file (default name: gunicorn.conf.py):

syslog = True
syslog_addr = "unix:///dev/log#dgram"
syslog_facility = "local0"
syslog_prefix = "myappname"

Syslog has 8 dedicated logging facilities that user applications can log to: local0 - local7. For each of my apps I’m using a different facility.

While developing it’s useful to see the logging output directly printed to the console. For that I’m using a different Gunicorn configuration file (I named it gunicorn.dev-conf.py) where I simply log everything to the console (access logs go to stdout and error logs to stderr):

errorlog = "-"
accesslog = "-"

To use the custom configuration file, I run gunicorn like this:

gunicorn -c gunicorn.dev-conf.py

Configure Syslog to write all output to a dedicated log file

Put this line in /etc/syslog.conf:

local0.*                                                /var/log/myappname

Be sure to put this line high enough in the configuration file so that there is no other rule that would catch your messages.

Then create an empty log file (important!) and restart syslogd daemon:

# touch /var/log/myappname
# rcctl restart syslogd

You can easily test whether server setup works with a following command:

# logger -p local0.info "Hello, world!"
# cat /var/log/myappname
Hello, world!

Conclusion

Overall I’m happy with the setup, especially how little is required to have a working logging setup. Having said that, there are several things I would like to improve:

1. Align logging formats for development and on the server. Gunicorn has some extra logic that would use different logging configuration when logging to the Syslog. This file should have all the answers…
2. Add millisecond or microseconds precision to the date format