@cpuguy83 what is the default path for docker volumes create ?

Same as creating a volume through "run -v", so /var/lib/docker/vfs/dir/<id>

@cpuguy83 oh thanks I didn't remebeer we changed this

@vieux That's actually not changed, or at least not changed in quite some time.

@cpuguy83 this is cool.
Maybe use 'volume' instead of 'volumes' would be better?

@zhgwenming Not really, because we are dealing with a collection of volumes.

I'd vote for using the singular, as in "docker volume" instead of "volumes". Most of the operations are on an individual volume, except "ls".

(It's the same argument for why some people name database tables as plural vs. singluar)

I'd actually prefer to have multiple top-level commands like "vol-create", "vol-delete", "vol-list", "vol-inspect", but that maybe too much like virsh for your liking.

I would expect docker volumes rm to have Unix unlink(2) semantics: dropping the ref from Docker itself to the volume (making it disappear from the list, and making nothing new able to link to it) but having the underlying store stick around until the last container using it is removed, at which point it gets removed automatically as well.

@tsutsu I sort of like the idea, but also feel it is somewhat surprising behavior. I think it falls into the realm of GC, which Docker does not handle itself at all right now.
I would prefer to keep things as simple as possible to start and if/when Docker implements a GC concept it can be applied to volumes.

@cpuguy83 Could we have, as part of the information returned by "docker volumes inspect <volume_name>", the containers that are using this volume?

@pchoco83 already there.

@cpuguy83 I just saw the "Aliases" info, thanks!

@cpuguy83 I compiled the docker binary from this branch in order to play with it.
I am not able to use "--volumes-from" from an existing volume:

#docker volumes inspect test-volume
[{
"Aliases": null,
"Created": "2014-10-28T18:09:22.583335347Z",
"Id": "db644ac06fc8e51de1bf433bb1bd5539805d6e2f6796946e7adc726cb5729158",
"IsBindMount": false,
"Name": "test-volume",
"Path": "/var/lib/docker/vfs/dir/db644ac06fc8e51.....946e7adc726cb5729158",
"Writable": true
}]

#docker run -d -P --volumes-from test-volume:/etc/mysql tutum/mysql
309aa2bdbaea256a2bd7e594f4d6caa5f3065725cd2f307b960fdcc13794a15c
[2014-10-28T18:16:13.680404629Z] [fatal] log.go:64 Error response from daemon: Cannot start container 309aa2bdbaea256a2bd7e594f4d6caa5f3065725cd2f307b960fdcc13794a15c: Container test-volume not found. Impossible to mount its volumes

Is this option not implemented yet?

@cpuguy83 I realized it works if using a "volume_alias" instead of the "volume_name".
I think it would be more usable if using "volume_name", thoughts?
Also, if the container that creates the alias is removed, the corresponding alias in the volume is gone and the volume becomes unusable.

@pchico83 No. --volumes-from is still the same basic behavior, except that now instead of grabbing all volumes from a container, you can grab just a single one, and have a custom mount point.
A volume alias is just a container name + volume mount path.

You can, however, use the name with -v.

@cpuguy83 Thanks! Is the "-v" option (using a volume name) available from the remote API?

@pchico83 The docker CLI only ever uses the remote API, so yes :)

@cpuguy83 Found it. Using "binds" when starting a container. Thanks!

@cpuguy83

Only one process can manage the pool. This process knows the transaction Id and manages pool transaction Id. Two processes can't do it.

Would such improvement be able to solve the other problem: that docker is not able to bind mount volumes into the container as a different UID and GID?

@cpuguy83

W.r.t complexity of dm, btrfs etc, it is already managed by graphdriver and image and container objects already use it. So we probably will not be dealing with new complexity if we extend volumes to make use of graph drivers other then VFS.

If extensions can't work on same pool where images and containers are stored, then it will not serve the use case I have been thinking of.

Also I am wondering why volume has to be a path. Why could it not be an object with a unique hash ID (like images) and then various metadata attached to it. And path will be just one of the metadata. If user specifies the path, then user is sort of responsible to making sure that volume is up before it is used by docker. Otherwise, it could be a docker managed volume and path will be automatically determined by docker (/var/lib/docker/volumes/mnt/).

Are there some patches somewhere for this extension based volume management. I want to play with it a bit and see what it will offer.

@rhvgoyal @cpuguy83

The previous patch for docker volume driver I mentioned separates pool/working directory for image and volume management, since I found it caused conflict on e.g. devId allocation which is per DeviceSet in device mapper(which I allocated another DeviceSet for the volume in the patch). If we want to use the same pool, I guess there would be a little more work need to be done.

However, the major problem still is, volume is assumed to be a directory currently, rather than an object. I don't think that assumption can hold in the future. We can keep path(as source and use vfs) for backward compatibility, but ultimately I think volume should be defined as: an object can be mounted to a path of guest container. Then we can do various things to the object.

Since we're already planning to have separate volume commands for docker, it would be so natural to have the volume as an object rather than merely a path. I think it would be necessary for the new volume driver able to use graphdriver as volume driver, rather than only vfs and host mount in the future.

I think volume should be defined as: an object can be mounted to a path of guest container. Then we can do various things to the object.

+1 Absolutely. IMO (and I think I mentioned that on other issues) a volume is storage where and how that storage is manager should be of no concern for the container. In a simple configuration, a volume could be stored on the host, in a directory. On a more advanced setup, this could be a remote volume or "you name it".

My ideal steps would be to (1) create/define a volume ("storage") using the desired driver. (2) connect the volume to the container.

It should be possible to manage the volume separately, for example to migrate or snapshot a volume, but all these could be dependent on the driver and (again) transparent to the container.

@yasker @thaJeztah

+1 to some of your ideas.

Few thoughts.

• I think we need a summary document which explains how this new volume architecture is going to look like. There seem to be information scattered in various threads and git trees but no single place
  explains it.
• In new scheme of things, are we targeting one global volume driver which will drive all volumes for
  that docker intance or there can be multiple volume drivers and it will become a per volume property. That means there might be some volumes backed by in-built graph driver, other volumes backed by plugin/extension volume driver. I feel making volumedriver a per volume property is much more flexible.
• Volume driver Plugin/extension seems to be just another way to plug in a volumedriver dynamically, but I guess core volume API will have to be defined by docker core and all plugin/extension will simply
  implement that API. I think this API should be pretty close to what graph driver is currently offering.
• I like the idea of treating volumes as object (like images) and then maintain metadata around it to differentiate between different kind of volumes. (Instead of just treating volume like a path). A call
  into volumedriver (volume.volumeDriver.Mount()) should just make volume available at certain mount point.
• Once should be able to use existing graphdrivers (btrfs, devicemapper etc) as volumedrivers.
• In fact I think if we allow per volume volume drivers, then one can have two different instances of
  devicemapper graph drivers. One plugged in one which is operating on its own separate thin pool
  and another one which is in-built and operating on pool where existing images are.

Those are some of the ideas. I think if we maintain a summary document which describes new volume architecture, it will help a lot.

Since this isn't really mergable today, it doesn't really make sense to keep it open, so I'm going to go ahead and close.

So a lot of this discussion is really great, but is probably better suited for docker-dev ML at this point, so it would be really great to continue there, or even on IRC for specific questions.

Thanks.

I like this.
Are there any plans to merge the above functionality? Is there any other active discussion somewhere on docker volume proposals?

@KodyKantor Some other bits of the code-base need to be handled first so this can be implemented with the suggested API.

@cpuguy83 This issue is referenced from https://docs.docker.com/userguide/dockervolumes/:

Note: Docker will not warn you when removing a container without providing the -v option to delete its volumes. If you remove containers without using the -v option, you may end up with “dangling” volumes; volumes that are no longer referenced by a container. Dangling volumes are difficult to get rid of and can take up a large amount of disk space. We’re working on improving volume management and you can check progress on this in pull request #8484

Since this is closed, where can I go to check the progress on improving volume management?

@laktak thanks for reporting; #14214 is a "tracking" issue for issues related to volume management. Are you perhaps interested in creating a pull-request to update the link in the documentation? The Markdown file that is used to generate that page in the documentation can be found here /docs/userguide/dockervolumes.md

The Markdown file that is used to generate that page in the documentation can be found here /docs/userguide/dockervolumes.md

@thaJeztah The text has already been updated (not by me), it's just not live.

I'm lost, was this functionality implemented or not?

@ovidiub13 no, it's not implemented (yet); see #14214

@thaJeztah @ovidiub13 In fact the volume management has already been implemented by @cpuguy83 at #14242 . It's in review now and I suppose it would be merged in v1.9.

@yasker yes, you're right; I intended to say "it's not merged yet"

Thank you. I'm looking forward to that release.

For those following this; this functionally is now implemented in #14242, which was just merged, and will be in docker 1.9

If you want to test it before that, nightly builds, build from master can be found at https://master.dockerproject.org