I kinda feel like by exposing these we are allowing people to play with fire

Maybe, but you are also eliminating lots of patch/pull requests. I would equate these in some ways to githooks.

These hooks would allow distributions to add distribution specific hooks and eliminate some of the RegisterMachine type pull requests.

We could always add something to docker info to indicate if there were any hooks present, so that your support people could require the hooks be removed and replicate the problem.

I also see this as fitting the model of Plugins that Docker has been talking about for many years.

@rhatdan could you explain why this should be separate binary?

@LK4D4 No reason, but I thought this was the way runc/libcontainer works. If you would prefer to extend docker to add the hooks, I could rework the patch. We originally wrote this for runc as a hook and just moved it to docker.

@rhatdan No, I mean it's anyway spawning hooks as separate programs. So, for me it looks like library for running hooks, more then hook itself.
But I'll take a closer look and try it, maybe it's more convenient.

@LK4D4 Yes, this allows running other hooks in a directory.

Is there way here to pass args to hooks? I think I'll need this (for closing named pipe for example).

Well hooks take args, so you can easily do this from runc. Also you can pass environment. Passing it from docker through to the dockerhooks, would be problematic. We are just passing the same data to every hook.

Another potential use case for docker hooks came up today. We were looking at sssd inside of a container, you might want to add a hook so that all containers got their identity information from the same container service. You could cause all containers to get a socket mounted into them for identity.

With the help of @mrunalp I have made dockerhooks into a simpler patch that no longer requires a separate program.

@mrunalp PTAL I simplified some more of the code and eliminated the hard coding of /usr/libexec/docker/hooks.d

Here is a little test script that I run to test if the hooks are getting called.

# cat /usr/libexec/docker/hooks.d/test.sh 
/usr/libexec/docker/hooks.d/test.sh 
#!/bin/sh
date >> /run/hooks.txt
echo "0:" $0 "1:" $1 >> /run/hooks.txt
read input
echo $input >> /run/hooks.txt

I agree with @jfrazelle. this seems a too low level hook to be qualified as a plugin and allowing user to play with fire. if we have a well-defined problem we could implement proper plugin mechanism for that defined problem and one could reason out the extension more clearly. networking is one such thing (and hence it is in CreateNetwork) which makes use of this and has a clearly defined plugin with a higher level plugin hook without exposing the lowest level hook.

I have given you use cases. RegisterMachine being our primary. We need a mechanism to hook the container into the machinectl of systemd, this works for us.

Another use case would be systemd itself. Allows us to setup a proper systemd environment if systemd is being run as the pid 1.

We also are looking at allowing a plugin for sssd so that /var/lib/sss could be setup to provide identity information into all containers.

I think you can document this as dangerous, but allow the distributions to decide how they want to plugin containers into their environments. And get Docker out of the business of having to review disrtibution specific changes to containers.

FWIW hooks are a standard pre+post op which are done in clustering for many reasons. Enabling it in as a core plugin, would also enable several higher order features.

3rd Party Auditing,
Initial/custom Health Check,
Custom Staging,
Custom Cleanup,
The list goes on...

xref 1: http://research.cs.wisc.edu/htcondor/manual/v8.5/4_4Hooks.html
xref 2: https://issues.apache.org/jira/browse/MESOS-2060
...

Almost every cluster management tool I know of develops some form of hooks, so why not leverage in the core.

/cc @vieux @thockin @tnachen

With @runcom vendored libcontainer patch this pull request finally passes the tests.

@LK4D4 and others PTAL. If you want to play with oci hooks you could look at these plugins.

https://github.com/sallyom/Register
https://github.com/mrunalp/hooks

ping @crosbymichael @LK4D4 what's the status here?

so t seems a bit unsafe to allow users to play in this low level way. It would be unfortunate if "docker" broke because a hook was actually broken. Plugins for volumes, networking, auth, etc (and any others that may be added) should be able to provide the functionality wanted by a hook but in a more controlled manner. So for now we are closing.

Those hooks do not give us the control to now when a container is starting or stopping. We need this to properly register with systemd machine ctl. How do I do this using one of these other plugins?

We also want to finally be able to support systemd within a container along with journald running properly. We need low level hooks for this. But you give us no alternatives?

@rhatdan - perhaps you could open an issue to discuss this? It appears that the proposed solution/work-around doesn't address your use-case and therefore more discussion seem appropriate - but unfortunately github won't let us re-open this PR :-(

We have enough now from runc perspective to handle our use cases. We have generate two oci hooks executables https://github.com/projectatomic/oci-register-machine and https://github.com/projectatomic/oci-systemd-hook. These hooks are shipping now in Fedora Rawhide and soon in F24. We are looking to release them to RHEL in May. oci-register-machine just informs systemd/machinectl that their is a container running on the system. This allows a user to execute machinectl and see docker, runc, systemd-nspawn, rkt, libvirt-lxc and VM's running using a single interface.

oci-systemd-hook sets up the container environment in a way that systemd can run as pid 1 in a non --privileged container.

@rhatdan We (over in NixOS) also would like to manage docker containers via systemd (as we leverage systemd extensively in our deterministic system configuration). Would you be interested in chatting some time to see if there's a way we could mutually benefit from our efforts? Don't know where the best place would be for that discussion -- if there's a mailing list I should hop on, I can do that -- or we can IRC (I idle on #nixos as cstrahan) / email.

@cstrahan Contact me at dwalsh@redhat.com or atomic-devel@projectatomic.io