Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The docker daemon API lets you create two networks with the same name #18864

Closed
vikstrous opened this issue Dec 23, 2015 · 19 comments · Fixed by #46251
Closed

The docker daemon API lets you create two networks with the same name #18864

vikstrous opened this issue Dec 23, 2015 · 19 comments · Fixed by #46251
Labels
area/networking kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. version/1.9

Comments

@vikstrous
Copy link
Contributor

The docker daemon API lets you create two networks with the same name. Having two networks with the same name breaks all kinds of things. It seems like the CLI client checks before creating a second network, but this is still a race condition. This should be prevented in the daemon.

docker version
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.5.1
 Git commit:   a34a1d5-dirty
 Built:        Sun Nov 22 00:15:15 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.5.1
 Git commit:   a34a1d5-dirty
 Built:        Sun Nov 22 00:15:15 UTC 2015
 OS/Arch:      linux/amd64
@thaJeztah thaJeztah added kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. area/networking labels Dec 23, 2015
@bboreham
Copy link
Contributor

@vikstrous did you notice there is a field CheckDuplicate in the API ?

https://docs.docker.com/engine/reference/api/docker_remote_api_v1.21/#create-a-network

I'd agree that having it default off is surprising.

@thaJeztah
Copy link
Member

ping @mavenugo @mrjana wdyt?

@vikstrous
Copy link
Contributor Author

Ah, okay, I did not see that. I can't think of a case where you would want to create two networks with the same name though. It clearly doesn't work.

@bboreham
Copy link
Contributor

What are the specific steps to demonstrate it not working? Do you have a test program?

You have to use the hex ID to refer unambiguously to a network, but as I recall it did work when I tried it.

@mrjana
Copy link
Contributor

mrjana commented Dec 23, 2015

@vikstrous @bboreham Allowing duplicate names is by design. Because network is primarily keyed based on a random ID and not on the name. The name is strictly a user-friendly alias to the network which is uniquely identified using ID. The checkDuplicate option is just there to provide a best effort checking of any networks which has the same name but it is not guaranteed to catch all name collisions. That is why it is not a default.

@vikstrous
Copy link
Contributor Author

Hmm... I had no idea that you can use the network's id (or that networks have IDs) when creating a container. We were using the name, so if a new network was created with the same name it would make the original network inaccessible. It looks like this is all intended behaviour and we should be using network IDs.

Imagine a tool that needs to run a container on a specific network and if it doesn't exist, create it. It can create it only by name, but if it creates a container on that network while referring to it by name, there's no guarantee that it's the same network it created. So it needs to store state about the network ID it created and somehow communicate this to other tools that need to use the same network.

I think it's useful to be able to create a network using some unique ID, but all of this behavior seems intentional, so you can close this issue.

@vikstrous
Copy link
Contributor Author

If the default in the docker client is to not allow creating two networks in the same name, maybe it should also be the default in the API and/or the dockerclient library. I guess this is something all libraries have to fix?

@thaJeztah
Copy link
Member

I still think we should look into this, and discuss if this is the desired behavior. I haven't tested yet what happens if I create two networks with the same name through the API, but from the docker CLI, there's no way to identify which network a container is connected to.

For example;

This container joined network foobar:

"Networks": {
            "foobar": {
                "EndpointID": "6dca6167ae2896c2276171b39fec35ba527b6deffa5521cb34ab188125944146",
                "Gateway": "172.18.0.1",
                "IPAddress": "172.18.0.2",
                "IPPrefixLen": 16,
                "IPv6Gateway": "",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "MacAddress": "02:42:ac:12:00:02"
            }
        }

And this one joined the same network through its ID

  "Networks": {
            "foobar": {
                "EndpointID": "5e5cd696f66bfeddea4b26b8466b9bb32b973be10b439e9c6094249fc40d821f",
                "Gateway": "172.18.0.1",
                "IPAddress": "172.18.0.3",
                "IPPrefixLen": 16,
                "IPv6Gateway": "",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "MacAddress": "02:42:ac:12:00:03"
            }
        }

The unique identifier of the network is not visible in the output here

@bboreham
Copy link
Contributor

@mrjana is this "best effort" nature of checkDuplicate documented anywhere?

The API docs say "Requests daemon to check for networks with same name".

@thockin
Copy link
Contributor

thockin commented Dec 25, 2015

Is there actually value to this semantic? Enough value ton outweigh the
obvious confusion? Containers don't allow the same name to be used more
than once, why should networks?
On Dec 24, 2015 6:28 AM, "Bryan Boreham" notifications@github.com wrote:

@mrjana https://github.com/mrjana is this "best effort" nature of
checkDuplicate documented anywhere?

The API docs say "Requests daemon to check for networks with same name".


Reply to this email directly or view it on GitHub
#18864 (comment).

@WeiZhang555
Copy link
Contributor

I feel like @vikstrous is right, we should have CheckDuplicate default on because it's more rational to NOT allow networks to have same name, that may bring more confusions.
This is a design problem, easy to modify in code.

@mrjana
Copy link
Contributor

mrjana commented Dec 25, 2015

@vikstrous If we have to make the API and cli consistent then they both should not check for duplicates because there is no guaranteed way to check for duplicates across a cluster of docker hosts. This is because the network object is stored by design with only the network ID as the key and name is just a user friendly alias. This schema allows in the future to assign multiple aliases to the same network.

@mrjana
Copy link
Contributor

mrjana commented Dec 25, 2015

@thaJeztah If the network ID does not show up in the inspect then we should fix that and that is all is needed to make the two networks usable (or avoid confusion)

@mrjana
Copy link
Contributor

mrjana commented Dec 25, 2015

@bboreham I will check the API docs later but if it doesn't mention that checkDuplicate is only best effort then that should be fixed in the docs.

@mrjana
Copy link
Contributor

mrjana commented Dec 25, 2015

@thockin The reason we have this semantic is because we want to adopt a schema which can pave the way in the future for a completely decentralized cluster of docker hosts (if scalability is needed). Already network objects are immutable for this reason(you can only create and delete networks but not modify). And because of the schema where the network is only truly identifiable by it's ID we can in the future cut over to a completely partition tolerant and decentralized cluster of docker hosts. That's the value.

Of course a stateful higher level orchestrator can implement a centralized solution and check for name collisions if it wants to.

@mrjana
Copy link
Contributor

mrjana commented Dec 25, 2015

@WeiZhang555 See the above comments for the reasons on why this design choice was made intentionally.

@vikstrous
Copy link
Contributor Author

@mrjana This sounds reasonable. Since you understand the problem well, you should make individual issues for each of the things discussed so we can make sure that the cli, api and docs are all consistent and accurate.

@doronp
Copy link
Contributor

doronp commented Feb 25, 2016

@thockin IMHO there is no reason for container names to be unique as well. Supporting Mutli tenancy for example will have to change that won't it?

suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Allow to set Docker network's CheckDuplicate option
to prevent name's collisions.
drone exec executes a local build, so make CheckDuplicate true is natural.

I will fix change of Dependencies after the following Pull Request has been merged.

cncd/pipeline#44
suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Allow to set Docker network's CheckDuplicate option
to prevent name's collisions.
drone exec executes a local build, so make CheckDuplicate true is natural.

Change of Dependencies should be fixed after the following Pull Request has been merged.

cncd/pipeline#44
suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Set Docker network's CheckDuplicate option to prevent name's collisions
at drone exec command.
drone exec executes a local build, so make CheckDuplicate true is natural.

Change of Dependencies should be fixed after the following Pull Request has been merged.

cncd/pipeline#44
suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Set Docker network's CheckDuplicate option to prevent name's collisions
at drone exec command.
drone exec executes a local build,
so it is natural to make CheckDuplicate true.

Change of Dependencies should be fixed after the following Pull Request has been merged.

cncd/pipeline#44
benhei pushed a commit to SAP-archive/cloud-s4-sdk-pipeline-docker that referenced this issue Oct 17, 2018
Docker does not ensure that network names are unique, so we need to
consider cases when they are not.

Related issues:

* moby/moby#33561
* moby/moby#18864 (comment)
milas added a commit to milas/compose that referenced this issue Jun 22, 2022
Within the Docker API/engine, networks have unique IDs, and the
name is a friendly label/alias, which notably does NOT have any
guarantees around uniqueness (see moby/moby#18864 [^1]).

During day-to-day interactive/CLI Compose usage, this is rarely
an issue, as Compose itself isn't creating networks concurrently
across goroutines. However, if multiple Compose instances are
executed simultaneously (e.g. as part of a test suite that runs
in parallel), this can easily occur.

When it does happen, it's very confusing for users and resolving
it via the `docker` CLI is not straightforward either [^2].

There's two primary changes here:
 * Pass `CheckDuplicates: true` to the Docker API when creating
   networks to reduce the likelihood of Compose creating duplicates
   in the first place
 * On `down`, list networks using a name filter and then remove
   them all by ID, as the Docker API will return an error if the
   name alias is used and maps to more than one network

Hopefully, this provides a better UX, since the issue should be
less likely to occur, and if it does, it can now be resolved via
standard Compose workflow commands.

[^1]: moby/moby#18864
[^2]: https://stackoverflow.com/questions/63776518/error-2-matches-found-based-on-name-network-nameofservice-default-is-ambiguo

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
milas added a commit to milas/compose that referenced this issue Jun 22, 2022
Within the Docker API/engine, networks have unique IDs, and the
name is a friendly label/alias, which notably does NOT have any
guarantees around uniqueness (see moby/moby#18864 [^1]).

During day-to-day interactive/CLI Compose usage, this is rarely
an issue, as Compose itself isn't creating networks concurrently
across goroutines. However, if multiple Compose instances are
executed simultaneously (e.g. as part of a test suite that runs
in parallel), this can easily occur.

When it does happen, it's very confusing for users and resolving
it via the `docker` CLI is not straightforward either [^2].

There's two primary changes here:
 * Pass `CheckDuplicates: true` to the Docker API when creating
   networks to reduce the likelihood of Compose creating duplicates
   in the first place
 * On `down`, list networks using a name filter and then remove
   them all by ID, as the Docker API will return an error if the
   name alias is used and maps to more than one network

Hopefully, this provides a better UX, since the issue should be
less likely to occur, and if it does, it can now be resolved via
standard Compose workflow commands.

[^1]: moby/moby#18864
[^2]: https://stackoverflow.com/questions/63776518/error-2-matches-found-based-on-name-network-nameofservice-default-is-ambiguo

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
milas added a commit to milas/compose that referenced this issue Jun 22, 2022
Within the Docker API/engine, networks have unique IDs, and the
name is a friendly label/alias, which notably does NOT have any
guarantees around uniqueness (see moby/moby#18864 [^1]).

During day-to-day interactive/CLI Compose usage, this is rarely
an issue, as Compose itself isn't creating networks concurrently
across goroutines. However, if multiple Compose instances are
executed simultaneously (e.g. as part of a test suite that runs
in parallel), this can easily occur.

When it does happen, it's very confusing for users and resolving
it via the `docker` CLI is not straightforward either [^2].

There's two primary changes here:
 * Pass `CheckDuplicates: true` to the Docker API when creating
   networks to reduce the likelihood of Compose creating duplicates
   in the first place
 * On `down`, list networks using a name filter and then remove
   them all by ID, as the Docker API will return an error if the
   name alias is used and maps to more than one network

Hopefully, this provides a better UX, since the issue should be
less likely to occur, and if it does, it can now be resolved via
standard Compose workflow commands.

[^1]: moby/moby#18864
[^2]: https://stackoverflow.com/questions/63776518/error-2-matches-found-based-on-name-network-nameofservice-default-is-ambiguo

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
debdutdeb pushed a commit to debdutdeb/compose that referenced this issue Jun 30, 2022
Within the Docker API/engine, networks have unique IDs, and the
name is a friendly label/alias, which notably does NOT have any
guarantees around uniqueness (see moby/moby#18864 [^1]).

During day-to-day interactive/CLI Compose usage, this is rarely
an issue, as Compose itself isn't creating networks concurrently
across goroutines. However, if multiple Compose instances are
executed simultaneously (e.g. as part of a test suite that runs
in parallel), this can easily occur.

When it does happen, it's very confusing for users and resolving
it via the `docker` CLI is not straightforward either [^2].

There's two primary changes here:
 * Pass `CheckDuplicates: true` to the Docker API when creating
   networks to reduce the likelihood of Compose creating duplicates
   in the first place
 * On `down`, list networks using a name filter and then remove
   them all by ID, as the Docker API will return an error if the
   name alias is used and maps to more than one network

Hopefully, this provides a better UX, since the issue should be
less likely to occur, and if it does, it can now be resolved via
standard Compose workflow commands.

[^1]: moby/moby#18864
[^2]: https://stackoverflow.com/questions/63776518/error-2-matches-found-based-on-name-network-nameofservice-default-is-ambiguo

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Aug 17, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was. It's been superseded
since then by Docker Swarm, which has a centralized control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Aug 17, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was. It's been superseded
since then by Docker Swarm, which has a centralized control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Aug 17, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was. It's been superseded
since then by Docker Swarm, which has a centralized control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Sep 8, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Sep 8, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Sep 8, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Sep 11, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Sep 11, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Sep 11, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Sep 11, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Sep 12, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
akerouanton added a commit to akerouanton/docker that referenced this issue Dec 20, 2023
Fixes moby#18864, moby#20648, moby#33561, moby#40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was. It's been superseded
since then by Docker Swarm, which has a centralized control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: moby#18864 (comment)
[2]: moby#18864 (comment)

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/networking kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. version/1.9
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants