Add overlayfs graph backend #7619
Conversation
|
Note: This targets v23 of overlayfs and was tested with the kernel from: Also, i noticed some kernel issues with removing directories from the lower layer in overlayfs when run on an btrfs filesystem, but it works fine when the backing filesystem is ext4. |
|
Cool! |
|
FYI, overlayfs has been merged, and will be on Linux 3.18 mainline |
|
@alexlarsson we need rebase! |
This backend uses the overlayfs union filesystem for containers plus hard link file sharing for images. Each container/image can have a "root" subdirectory which is a plain filesystem hierarchy, or they can use overlayfs. If they use overlayfs there is a "upper" directory and a "lower-id" file, as well as "merged" and "work" directories. The "upper" directory has the upper layer of the overlay, and "lower-id" contains the id of the parent whose "root" directory shall be used as the lower layer in the overlay. The overlay itself is mounted in the "merged" directory, and the "work" dir is needed for overlayfs to work. When a overlay layer is created there are two cases, either the parent has a "root" dir, then we start out with a empty "upper" directory overlaid on the parents root. This is typically the case with the init layer of a container which is based on an image. If there is no "root" in the parent, we inherit the lower-id from the parent and start by making a copy if the parents "upper" dir. This is typically the case for a container layer which copies its parent -init upper layer. Additionally we also have a custom implementation of ApplyLayer which makes a recursive copy of the parent "root" layer using hardlinks to share file data, and then applies the layer on top of that. This means all chile images share file (but not directory) data with the parent. Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
alexlarsson
force-pushed
the
overlayfs
branch
from
9ab9749
to
453552c
Compare
|
Rebased to master. I'm also looking at making it work with selinux. Researching the posibilities atm. |
|
@alexlarsson thanks for the rebase. What are the existing tasks that still need to happen for this PR to be merged? Is there something that you need from us to help? |
|
@crosbymichael It works as-is, but not with selinux. I'm trying to figure out a way to make it support selinux, and the solution may involve shuffling around the directories a bit in the backend directory, so maybe we should wait a bit before merging. |
|
Also, there are some issues with overlayfs on top of other filesystems than ext4, the other filesystems need whiteout support similar to: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/fs?id=cd808deced431b66b5fa4e5c193cb7ec0059eaff I'm hopeful this will happen for the common filesystems before 3.18 release though. |
|
The approach and general naming of things lgtm! |
|
Assigned @unclejack to see this one through. Looks awesome! |
|
@unclejack lmk if you want me to try it out, i was going to build the RC kernel anyways |
|
I've tested this graph driver backend. This overlayfs backend seems to be working just fine. LGTM |
|
I tested as well and it's pretty swell LGTM |
|
@unclejack @jfrazelle did you tested over btrfs? |
|
yes, tested over btrfs |
|
@jfrazelle ok, then I'll try too :D |
|
\o/ On Thu, Nov 13, 2014 at 8:56 PM, Alexandr Morozov notifications@github.com
|
|
I believe btrfs doesn't yet implement RENAME_WHITEOUT, so it will fail in some situations. |
|
Also tested, but on ext4. Looks great! |
|
I'm not worried about issues if you run this on top of btrfs. I wouldn't expect it to magically work as we all know, overlay filesystems on top of overlay filesystems or the like usually end in flames. |
| if strings.Contains(s.Text(), "overlayfs") { | ||
| return nil | ||
| } | ||
| } |
There was a problem hiding this comment.
perhaps a log.Debugf("overlayfs not found") or similar, otherwise there is no distinction of error from overlayfs support even being compiled into docker itself
There was a problem hiding this comment.
@vbatts Do you think we need to change this before merging this graph driver? We could send a PR afterwards to make this change unless we decide this has to be changed before the merge.
|
besides a non-blocking nit, LGTM |
|
Ok, I'll merge this PR. We can address @vbatts' comment in a PR. Thanks, @alexlarsson! |
|
Yes! Congrats @alexlarsson :-) |
|
just a FYI i see this in the containers mount table using this driver:
|
|
Nice! Does this need a mention in the documentation (as "experimental")? |
|
I tried this overlayfs driver on Ubuntu 14.04.1 with ext4 file system as the root of the Docker runtime. I got this error "Error response from daemon: invalid argument". if I upgrade kernel to 3.18 rc3, then it works fine. Is overlayfs kernel support required? Thanks |
|
@helenxu1221 correct. overlayfs is added in linux-3.18, which is not even released yet. The error of docker-supported-ness will get clearer soon. Thanks for testing. |
|
To test out overlayfs in boot2docker: http://files.container42.com/boot2docker/builds/1.3.1-dev-overlayfs/boot2docker.iso |
|
With overlayfs, is there a way to set container disk limit like dm.basesize option? Thanks |
|
@helenxu1221 No, there's no such setting because this is similar to aufs. |
|
so there is no way to limit the container size? even from cgroup level. |
|
No, there isn't. |
based on moby#7619 (comment) Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
|
This needs to be updated since the fs name changed from overlayfs to overlay upstream: |
based on moby#7619 (comment) Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
|
Will this allow multiple Dockerfiles to be merged together? |
|
@marcellodesales This is support for the Overlay filesystem Not really anything to do with Dockerfiles or any change in how Docker itself works. |
|
@cpuguy83 My bad! I will read the docs... :) |
|
@alexlarsson is there a good primer out there on how to use Docker with OverlayFS? I've got the 3.18 kernel installed, but am confused about what needs to happen before docker can use OverlayFS. |
|
Adding the daemon flag -s overlay is all you need On Thursday, December 25, 2014, Vincent Woo notifications@github.com
|
|
@jfrazelle Hm, maybe you can help me out with this then. I'm running 3.18: And have the following block in But when docker tries to start: |
|
It's overlay not overlayfs, we changed the name when the kernel did On Thursday, December 25, 2014, Vincent Woo notifications@github.com
|
|
oh christ I'm an idiot |
|
body { |
|
No worries! It was overlayfs in this PR and we switched it after ;) On Thursday, December 25, 2014, Vincent Woo notifications@github.com
|
This backend uses the overlayfs union filesystem for containers
plus hard link file sharing for images.
Each container/image can have a "root" subdirectory which is a plain
filesystem hierarchy, or they can use overlayfs.
If they use overlayfs there is a "upper" directory and a "lower-id"
file, as well as "merged" and "word" directories. The "upper"
directory has the upper layer of the overlay, and "lower-id" contains
the id of the parent whose "root" directory shall be used as the lower
layer in the overlay. The overlay itself is mounted in the "merged"
directory, and the "work" dir is needed for overlayfs to work.
When a overlay layer is created there are two cases, either the
parent has a "root" dir, then we start out with a empty "upper"
directory overlaid on the parents root. This is typically the
case with the init layer of a container which is based on an image.
If there is no "root" in the parent, we inherit the lower-id from
the parent and start by making a copy if the parents "upper" dir.
This is typically the case for a container layer which copies
its parent -init upper layer.
Additionally we also have a custom implementation of ApplyLayer
which makes a recursive copy of the parent "root" layer using
hardlinks to share file data, and then applies the layer on top
of that. This means all chile images share file (but not directory)
data with the parent.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson alexl@redhat.com (github: alexlarsson)