Re: Supervising a pipeline? <SOLVED for this use case>

From: Steve Litt <slitt_at_troubleshooters.com>
Date: Sat, 26 Dec 2015 19:34:00 -0500

On Sat, 26 Dec 2015 18:25:22 +0100
Laurent Bercot <ska-supervision_at_skarnet.org> wrote:

> On 2015-12-26 18:09, Steve Litt wrote:
> > #!/bin/sh
> > exec /usr/bin/inotifywait /dev/disk
> > | /usr/local/bin/automounter.py
>
> You can't supervise a pipeline per se; you need to supervise
> both processes in the pipeline independently, and make sure the pipe
> isn't broken when one of them dies.

I've verified that the preceding paragraph to be true. Killing the
Python program left the inotifywait running, and the service
non-restarted. That's unacceptable.

>
> So, have "exec inotifywait /dev/disk" as foobar/run, and have
> "exec automounter.py" as foobar/log/run. This will work with
> daemontools, runit and s6. (You can accomplish the same goal with
> perp and nosh too; the syntax will just be different.)

:-)

Congrats Laurent: You've suggested a kludge that the King of Kludges,
Steve Litt, cannot abide by. I understand it would probably work, but
I'm too much of a prude to use a facility, meant for logging, in this
way. Nice one though: The kludgicity is a thing of panoramic beauty. If
it had been my idea I might have done it just for that reason :-)


>
> Alternatively, you could use s6-rc and create the "inotifywait" and
> "automounter" longrun services in a pipeline; your compiled database
> would then include instructions to set up the supervised pipeline
> for you. This is more complex to set up than just using the integrated
> pipe management in svscan and runsvdir, but it's also more powerful,
> because you can pipeline an arbitrary number of processes that way
> (this is also what nosh does).

I'll keep the preceding in mind when contemplating:

exec a | b | c | d | e

But I can't begin to imagine what would be in the "compiled database"
to which you refer. I can't actually do this for the following reasons:

1) I have absolutely no idea how.

2) Documentation for deployment would be a nightmare.

3) My intended audience would not only laugh, but use such a setup as a
   reason not to use my automounter.


At this point I should mention the pipexec described by post-sysv. This
looks promising in a different situation, but it would be a dependency
that would belie my automounter's claim to "simplicity."

So what I did was something like this, in the Python program:

====================================================
proc = subprocess.Popen(['/usr/bin/inotifywait', \
  '-m', '-r', '/dev/disk/by-id'], \
  stdout=subprocess.PIPE, bufsize=1)

EOF = False
while not EOF:
    line = proc.stdout.readline().decode('utf8')
    if line == '':
        EOF = True
    else:
        line = line.strip()
        process_line(line)

print('\nCaller loop done, aborting program!!!!')
====================================================

The runit run command becomes:

exec /usr/local/bin/amounter.py

While not as kludgificent as using the logging facility, the preceding
does have a certain charmful kludgicity.

And, as tested with runit, if amounter.py dies, they both die and the
service gets rerun, and if inotifywait dies, they both die and the
service gets rerun.

Thanks everybody!

SteveT

Steve Litt
November 2015 featured book: Troubleshooting Techniques
     of the Successful Technologist
http://www.troubleshooters.com/techniques
Received on Sun Dec 27 2015 - 00:34:00 UTC

This archive was generated by hypermail 2.3.0 : Sun May 09 2021 - 19:44:19 UTC