User namespaces: graduate from experimental #19187

estesp · 2016-01-08T15:55:51Z

Moves experimental user namespaces to master in preparation for the 1.10 release. Adds documentation to regular docker daemon command docs. Properly warns of incompatible flags for namespace sharing + privileged.

Closes: #18448
Closes: #18430
Closes: #18247

estesp · 2016-01-08T15:59:40Z

ping @thaJeztah : I spent some extra time trying to make the docs more readable/clear. Feel free to start reviewing/providing feedback as I think the docs may need more help than the code, which is basically just a migration of the flag and settings.

thaJeztah · 2016-01-08T16:02:14Z

ping @moxiegirl as well 😄

vdemeester · 2016-01-08T16:05:11Z

😻

jessfraz · 2016-01-08T17:13:00Z

yayayayayayayayayayayayyayaaya! LGTM

LK4D4 · 2016-01-08T17:15:41Z

yolo
LGTM

icecrime · 2016-01-08T17:16:39Z

Looks YOLO To Me 👍

Adds the `--userns-remap` flag to the master build Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)

estesp · 2016-01-08T20:09:04Z

@jfrazelle just FYI--updated the patch to stop hack/make.sh from adding experimental build tag when DOCKER_REMAP_ROOT is set, given it will now be "testable" from master. Let me know if you see any issues with that, but the CI userns run should verify that's correct.

jessfraz · 2016-01-08T20:13:20Z

sweet thats perfect thanks!

On Fri, Jan 8, 2016 at 12:10 PM, Phil Estes notifications@github.com
wrote:

@jfrazelle https://github.com/jfrazelle just FYI--updated the patch to
stop hack/make.sh from adding experimental build tag when
DOCKER_REMAP_ROOT is set, given it will now be "testable" from master.
Let me know if you see any issues with that, but the CI userns run
should verify that's correct.

—
Reply to this email directly or view it on GitHub
#19187 (comment).

thaJeztah · 2016-01-08T22:08:57Z

@estesp don't forget to update the daemon man page as well, to include the new flag; https://github.com/docker/docker/blob/master/man/docker-daemon.8.md

HackToday · 2016-01-11T01:01:08Z

@thaJeztah do you know how to test it when it was not merged ? Do we have binary build for that ? Or need I cherry-pick these code in my repo and build binary by myself ? Thanks

vdemeester · 2016-01-11T07:34:50Z

@HackToday You can try out a pull request by checking it out using git and then build it (using make binary for example 😉)

HackToday · 2016-01-11T08:58:37Z

Thanks @vdemeester I tried this clone
https://github.com/estesp/docker/tree/lets-do-this

Since I did not figure it out just cherry-pick easy ways to do that :(

HackToday · 2016-01-11T09:55:51Z

Found following potential issues:

use host with --uts=host or --ipc=host failed with following strange error message:

root@003d1da3734a:/go/src/github.com/docker/docker# docker run -it --uts=host myhtop pidtest-1
mount: permission denied
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied
root@003d1da3734a:/go/src/github.com/docker/docker# docker run -it --ipc=host myhtop pidtest-1
mount: permission denied
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied

for --ipc use container:ID

root@003d1da3734a:/go/src/github.com/docker/docker# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5040226021d9 busybox "sh" 23 minutes ago Up 23 minutes testok
2dc6bb7c5ebb busybox "sh" 25 minutes ago Up 25 minutes goofy_northcutt
root@003d1da3734a:/go/src/github.com/docker/docker#
root@003d1da3734a:/go/src/github.com/docker/docker#
root@003d1da3734a:/go/src/github.com/docker/docker# docker run --name testsecond -it --ipc=container:testok busybox sh
docker: Error response from daemon: Container command could not be invoked..
root@003d1da3734a:/go/src/github.com/docker/docker# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
98268acac542 busybox "sh" 8 seconds ago Created testsecond
5040226021d9 busybox "sh" 23 minutes ago Up 23 minutes testok
2dc6bb7c5ebb busybox "sh" 25 minutes ago Up 25 minutes goofy_northcutt

And the container was created.

HackToday · 2016-01-11T10:02:27Z

And another issue:

When run with docker run --read-only -v /icanwrite busybox sh still have long go trace message like below:

5: main
Package: main
File: docker.go@18

6: main
Package: runtime
File: proc.go@111

7: goexit
Package: runtime
File: asm_amd64.s@1721
docker: Error response from daemon: Cannot start container c8bd26e061fe5d4b229581c912b0204c0dee4b3bebf82e72940895591bf74f09: [9] System error: operation not permitted.

It seems still an issue mentioned like this one #18430 (although the old one is about pid namespace)

HackToday · 2016-01-11T10:05:24Z

daemon/daemon_unix.go

+			return warnings, fmt.Errorf("Cannot share network namespaces with user namespaces enabled.")
+		}
+		if hostConfig.PidMode.IsHost() {
+			return warnings, fmt.Errorf("Cannot use the host PID namespace with user namespaces enabled.")


Suggest replace use with share, as keep consistent above network namespace error message.

estesp · 2016-01-11T16:39:00Z

And another issue:

When run with docker run --read-only -v /icanwrite busybox sh still have long go trace message like below:

I have added an early error similar to the namespace sharing errors when using --read-only=true. The docs mention this is not possible with user namespaces, but adding the error makes it much clearer to the user. Thanks.

estesp · 2016-01-11T16:50:56Z

@thaJeztah I have made the modifications you suggested as well as added the additional info to the docker-daemon man page. Please let me know if the man page additions are sufficient of if the more complete detail from the markdown docs should be added here as well.

@HackToday I have made corrections to the docs per your suggestions and added an error message when --read-only is in use (to prevent the long complicated Go dump output). Your other issues (with shared ipc/uts) seem to be related to something your specific container is doing at startup? If I run a simple "busybox" or "ubuntu" image, I don't get these errors. I can look into it further, but maybe you can try with a standard image and then understand what your "myhtop" container is trying to do

jamtur01 · 2016-01-12T00:43:52Z

docs/reference/commandline/daemon.md

+## Daemon user namespace options
+
+Linux kernel [user namespace support](http://man7.org/linux/man-pages/man7/user_namespaces.7.html) provides additional security by enabling
+a process--and therefore a container--to have a unique range of user and


single dashes if you must use dashes.

- and therefore a container -

jamtur01 · 2016-01-12T00:49:14Z

Nice Docs - minor edits.

estesp · 2016-01-12T01:20:37Z

Thanks for the docs review @jamtur01. I have updated the PR per your comments.

jamtur01 · 2016-01-12T01:28:39Z

One minor fix but otherwise Docs LGTM. Thanks!

estesp · 2016-01-12T01:43:25Z

Great! @jamtur01 I've used "superuser" instead, and the PR is updated. Thanks!

This prevents strange errors and clarifies which namespace options are incompatible with user namespaces (at this time). Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)

Remove the experimental docs for user namespaces and add similar content to the `docker daemon` command documentation. Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)

cyphar · 2016-01-12T02:26:45Z

I think that #19251 should be considered before we make this non-experimental (quite a few systems don't have the latest version of shadow).

HackToday · 2016-01-12T05:10:03Z

LGTM

cyphar · 2016-01-12T05:38:32Z

@HackToday "LGTM"s should only be used by Docker maintainers (they are used by other maintainers to figure out whether there is a consensus on a feature being merged). It's just confusing to have non-maintainers LGTM things.

HackToday · 2016-01-12T06:50:41Z

@cyphar sure, Ignore LGTM, I will +1 about this

HackToday · 2016-01-12T06:50:59Z

👍

cpuguy83 · 2016-01-12T16:32:26Z

ping @thaJeztah for docs signoff

thaJeztah · 2016-01-12T17:34:08Z

docs LGTM, let's merge!

User namespaces: graduate from experimental

tiborvass · 2016-01-12T17:38:02Z

\o/

GordonTheTurtle added the status/0-triage label Jan 8, 2016

estesp added this to the 1.10.0 milestone Jan 8, 2016

estesp added status/2-code-review and removed status/0-triage labels Jan 8, 2016

estesp force-pushed the lets-do-this branch from 7fb3148 to 8191d2a Compare January 8, 2016 16:45

icecrime added status/3-docs-review and removed status/2-code-review labels Jan 8, 2016

Move userns support out of experimental into master

557c7cb

Adds the `--userns-remap` flag to the master build Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)

estesp force-pushed the lets-do-this branch from 8191d2a to 195476e Compare January 8, 2016 20:06

thaJeztah added docs/revisit impact/changelog labels Jan 10, 2016

HackToday reviewed Jan 11, 2016
View reviewed changes

estesp force-pushed the lets-do-this branch from 195476e to eb09d10 Compare January 11, 2016 16:47

jamtur01 reviewed Jan 12, 2016
View reviewed changes

estesp force-pushed the lets-do-this branch from eb09d10 to b85f390 Compare January 12, 2016 01:19

estesp force-pushed the lets-do-this branch from b85f390 to 9ac2742 Compare January 12, 2016 01:41

estesp added 2 commits January 11, 2016 20:50

Properly report conflicting namespace options when using userns

d5743a3

This prevents strange errors and clarifies which namespace options are incompatible with user namespaces (at this time). Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)

Add daemon documentation on user namespaces feature

cc63db4

Remove the experimental docs for user namespaces and add similar content to the `docker daemon` command documentation. Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)

estesp force-pushed the lets-do-this branch from 9ac2742 to cc63db4 Compare January 12, 2016 01:56

cyphar mentioned this pull request Jan 12, 2016

userns: add fallback sub{uid,gid} #19251

Closed

thaJeztah added a commit that referenced this pull request Jan 12, 2016

Merge pull request #19187 from estesp/lets-do-this

c72be04

User namespaces: graduate from experimental

thaJeztah merged commit c72be04 into moby:master Jan 12, 2016

estesp deleted the lets-do-this branch January 12, 2016 18:06

sdurrheimer mentioned this pull request Jan 12, 2016

Add zsh completion for 'docker daemon --userns-remap' #19272

Merged

albers mentioned this pull request Jan 26, 2016

bash completion for docker daemon --userns-remap #19731

Merged

bryanlatten mentioned this pull request Feb 1, 2016

Create a user for the container behance/docker-php#27

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

User namespaces: graduate from experimental #19187

User namespaces: graduate from experimental #19187

estesp commented Jan 8, 2016

estesp commented Jan 8, 2016

thaJeztah commented Jan 8, 2016

vdemeester commented Jan 8, 2016

jessfraz commented Jan 8, 2016

LK4D4 commented Jan 8, 2016

icecrime commented Jan 8, 2016

estesp commented Jan 8, 2016

jessfraz commented Jan 8, 2016

thaJeztah commented Jan 8, 2016

HackToday commented Jan 11, 2016

vdemeester commented Jan 11, 2016

HackToday commented Jan 11, 2016

HackToday commented Jan 11, 2016

HackToday commented Jan 11, 2016

HackToday Jan 11, 2016

estesp Jan 11, 2016

estesp commented Jan 11, 2016

estesp commented Jan 11, 2016

jamtur01 Jan 12, 2016

jamtur01 commented Jan 12, 2016

estesp commented Jan 12, 2016

jamtur01 commented Jan 12, 2016

estesp commented Jan 12, 2016

cyphar commented Jan 12, 2016

HackToday commented Jan 12, 2016

cyphar commented Jan 12, 2016

HackToday commented Jan 12, 2016

HackToday commented Jan 12, 2016

cpuguy83 commented Jan 12, 2016

thaJeztah commented Jan 12, 2016

tiborvass commented Jan 12, 2016

User namespaces: graduate from experimental #19187

User namespaces: graduate from experimental #19187

Conversation

estesp commented Jan 8, 2016

estesp commented Jan 8, 2016

thaJeztah commented Jan 8, 2016

vdemeester commented Jan 8, 2016

jessfraz commented Jan 8, 2016

LK4D4 commented Jan 8, 2016

icecrime commented Jan 8, 2016

estesp commented Jan 8, 2016

jessfraz commented Jan 8, 2016

thaJeztah commented Jan 8, 2016

HackToday commented Jan 11, 2016

vdemeester commented Jan 11, 2016

HackToday commented Jan 11, 2016

HackToday commented Jan 11, 2016

HackToday commented Jan 11, 2016

HackToday Jan 11, 2016

Choose a reason for hiding this comment

estesp Jan 11, 2016

Choose a reason for hiding this comment

estesp commented Jan 11, 2016

estesp commented Jan 11, 2016

jamtur01 Jan 12, 2016

Choose a reason for hiding this comment

jamtur01 commented Jan 12, 2016

estesp commented Jan 12, 2016

jamtur01 commented Jan 12, 2016

estesp commented Jan 12, 2016

cyphar commented Jan 12, 2016

HackToday commented Jan 12, 2016

cyphar commented Jan 12, 2016

HackToday commented Jan 12, 2016

HackToday commented Jan 12, 2016

cpuguy83 commented Jan 12, 2016

thaJeztah commented Jan 12, 2016

tiborvass commented Jan 12, 2016