s6
Software
www.skarnet.org

The s6-log program

s6-log is a reliable logging program with automated log rotation, similar to daemontools' multilog, with full POSIX regular expression support.

Interface

     s6-log [ -q | -v ] [ -b ] [ -p ] [ -t ] [ -e ] logging script

s6-log reads and compiles logging script to an internal form. Then it reads its standard input, line by line, and performs actions on it, following the script it is given. It does its best to ensure there is never any log loss. It exits cleanly when stdin closes or when it receives SIGTERM.

Options

Logdirs

A logdir (logging directory) is a place where logs are stored. s6-log can be scripted to write into one or more logdirs.

A logdir may contain the following files:

Rotation

In a logdir, selected lines are appended to the current file. When current becomes too big, a rotation happens. The current file will be possibly processed, then it will become an archived log file named @timestamp.s, where timestamp, a TAI64N timestamp, is the absolute time of the rotation. If there are too many archived log files in the logdir, the older ones are then suppressed. Logging then resumes, to a brand new current file.

You can use this mechanism to ensure that your logs never fill up the available disk space, for instance: something that neither syslogd, nor syslog-ng, nor rsyslog offers.

Processors

A processor script can be set for every logdir. When a rotation occurs, current (which has then been renamed previous) is fed to processor's stdin, and processor's stdout is saved and archived. processor can also read the state file on its fd 4; what it writes to its fd 5 will be saved as the next state file, for the next rotation.

Processors should not background themselves: s6-log considers the processing done when its processor direct child dies. Processors should exit 0 on success and nonzero on failure; if a processor fails, s6-log will try it again after some cooldown time.

Processors make s6-log Turing-complete by allowing you to use any external program to handle log files that are going to be archived.

Logging script syntax

When starting up, s6-log reads its arguments one by one; this argument sequence, or directive sequence, forms a logging script which tells s6-log what to log, where, and how.

Every directive can be a selection directive, a control directive or an action directive. A valid logging script always contains at least one action directive; every action directive can be preceded by zero or more selection or control directives. s6-log will exit 100 if the script is invalid. If it can process the script but the last directive is not an action directive, s6-log will emit a warning.

Selection directives

These directives tell s6-log whether to select or deselect lines it reads from stdin; actions will only happen on selected lines. By default, every line is selected.

Control directives

These directives tune s6-log's behaviour for the next actions.

Action directives

These directives determine what s6-log actually does with the selected lines.

Signals

Examples

     s6-log -bt n20 s1000000 /var/log/services/stuff

Logs all of stdin, prepending every line with a timestamp, into the /var/log/services/stuff logdir, with a maximum archive of 20 log files of 1 MB each; makes sure every line has been written before reading the next one.

     s6-log -t n30 E500 - +fatal: e - +^STAT =/var/log/foobard/status f s10000000 S15000000 !"gzip -nq9" /var/log/foobard

Why use execlineb to interpret the "processor" string ?

Because it is exactly what execlineb is for.

Why have another logging mechanism ?

Because the syslog mechanism and all its implementations (save one) suck. I'm not being judgmental; I'm just stating the obvious.

The syslog design is flawed from the start

When asked why he started rsyslog, Rainer Gerhards came up with a lot of hand-waving and not a single word about technical points. There is a reason for that: rsyslog is forked from sysklogd! So, no matter how many bells and whistles are added to it, it still suffers from the same basic flaws.

The problem with syslogd does not come from such or such implementation. The problem comes from syslog's design in the first place.

syslog is slow, it's unsafe, and it's incomplete. The only reason people use it is because it's historical, it exists, and there hasn't been any serious alternative yet, except maybe multilog, which s6-log improves upon.

A not-so-modest proposal: the logging chain

Unix distributions already do this to some extent, but it's at best unclear where the logs go for any given program.

So, given a program, where are its logs sent ?

What does s6-log have to do with all this ?

In a logging chain situation, every service must have its own logger. To avoid syslogd's design mistakes, one logger process per service must be run. s6-log fits that role. Using s6-log as your one-stop logger offers the following benefits:

You're wrong about being as powerful as syslogd: s6-log does not do remote logging.

You mean you want to send, live, every log line over the network via UDP ? You can't be serious.

Do yourself a favor and use s6-log to write log lines to a logdir, with a processor script that sends files-being-archived to the network, possibly after compressing them. More reliability, less log lines lost, less network traffic, better engineering. If you have no disk to even write the current files to, write to a small RAM filesystem.

If you have to log stuff live via the network, you do not need any local logging software. You don't even need syslogd. Just filter your stderr via some grep that selects lines for you, then sends them to a network socket. A trivial shell script, or execline script, can do that for you.

Do not insist on using syslogd. It does nothing magical, and nothing that can't be done in a simpler way using simpler tools. Major Unix distributions have finally caught the supervision train and abandoned the System V init model; soon, hopefully, they'll also catch the logging chain train and abandon the syslogd model.