Thanks so much @mapuri! I just gave docs/reference/commandline/build.md a quick read, and it seems to perfectly match what has been discussed <3

Personal opinion: i don't like the --description flag on ARG :S

This looks pretty good, but wondering should be force a default value for ARGs... this would also simplify the syntax and pretty much match ENV.

@tiborvass

regarding --description flag on ARG, are you saying you don't see need for it? Or does it need to be renamed? I assume former. If so as I understand the need for this flag comes from the desire to document and persist info about the ARG like it's expected use, purpose etc. Personally I feel it is good to have but I am fine removing it. It's an optional flag so the author still has a choice to not use it.

@cpuguy83

wondering should be force a default value for ARGs

An optional default value for ARG offers the flexibility to not expand or pass ARG around if it is not passed at all by --build-arg i.e. it just acts as a placeholder of values that can be accepted as a --build-arg but doesn't imply a value to be assumed on it's own. I think this comment from @duglin #9176 (comment) brings out a need for this well.

Wondering how we should go about this change, given that the ROADMAP.md mentions we won't be accepting changes to the Dockerfile format. I know other proposals were turned down (or put on hold) because of that, and accepting this PR may lead to complaints.

@thaJeztah I know, but the number of people complaining about that missing feature is just ridiculously high, and we have been discussing this in grouped review: I think we should take it.

Can't wait this one to be merged, this is effectively a huge improvement for builds.

Personally I'd still prefer to require a default for an arg, then later we can add the flag if it is really truly necessary.
Also agree we don't need --description since these can be documented with comments... and again if we do find it is necessary we can add the flag later.

What do we need to do to move this forward?

re: default value - I like the idea of a default value but I would make it optional otherwise we have no way of representing "not set at all" - the best we could do is set it to "" which isn't the same thing.

ARG foo    # defines 'foo' but no default value, which means it doesn't show up in "RUN env"
           # output unless --build-arg is used
ARG foo=bar   # foo set to 'bar' and "RUN env" shows "foo=bar"
ARG foo bar   # same as previous

and in all 3 cases, --build-arg foo=cat would result in "RUN env" showing "foo=cat"

re: --description - I actually see value in this. If there are 3rd party tools that wish to be able to prompt people for info prior to doing a build then this is a nice programmatic way to extract info about the arg. I do agree that we could technically add it later if we wanted, but I'm +1 for keeping it. However, its really only useful if that info is stored in the image since we wouldn't be able to query for inherited ARG's descriptions w/o it. Haven't looked at the code to know if this is saved in the image or not. @mapuri ?

Couple of comments:
1 - the idea of an image author being able to, in essence, say "here are some variables that a builder may want to set/override" is nice. Documenting this as part of an image is goodness - especially if we include the "description" field in the image metadata - which should be possible based on looking at the code.
2 - right now if someone specifies a "build-arg" that isn't also named via a "ARG" then it is ignored. We should either flag it as an error so the user knows they specified useless info, or we should use it anyway. I'd prefer to use it.

Either way we go on # 2 I'm ok with the design and suggest we move it to code review.

rebase needed

I'd prefer to error out, I think the point of the design is to make sure args are specified in the Dockerfile.

thanks for the review folks.

@duglin

1 - the idea of an image ...
... which should be possible based on looking at the code.

Yes the ARG description is stored as part of image metadata (i.e. they can queried using docker inspect).

2 - right now if someone specifies a "build-arg" ...
... We should either flag it as an error so the user knows they specified useless info,

Yeah, I had thought about this. I intentionally left the behavior to ignore as I couldn't think of a better/natural handling in a case where an ARG is defined at a line after it is accessed:

Just to show it with a made up scenario where I don't think it is ok to fail:

$ docker build --build-arg user=foo - << MARK
USER ${user:-bar}         <<< right now, this just evaluates to 'bar' and behaves in similar way USER primitive will behave today. Note we don't want to use the `-build-arg` value yet as it is not defined till this point. Similarly, it doesn't seem ok to error this build out as well.
ARG user
USER $user         <<< Since now ARG is defined, this shall use the value passed from cli i.e. foo
MARK

or we should use it anyway. I'd prefer to use it.

Since we are introducing ARG primitive to provide a list of allowed args I think it might not be ok to use the build-args until the point they are defined in Dockerfile.

@mapuri I agree that your second scenario should not fail, but that's not the case I was thinking of. If we do want to error on the case of "unknown --build-args", then by the time we reach the end of the Dockerfile, any arg that was specified via "--build-arg" that was not used should cause an error to be generated.

I noticed that the default build args are saved in the image - I don't think we should do that for a couple of reasons:
1 - it'll add extra noise to images that don't know or care about build-args
2 - the list of default build-args should be configurable based on the install/admin. Meaning, unless someone explicitly used ARG to add "http_proxy" then it should be up to the admin of the docker daemon to decide if they want "http_proxy" to be available via --build-args. So, let's add a config field to allow admins to define this. Absence of the property means the builder will use the default list, but presence of the property with a value (or empty list) means they don't want any default ones to be available except the ones mentioned.
3 - related to # 2, over time the list of default args that Docker has may change (grow or shrink) and I wouldn't want to force everyone to have to rebuild all of their images just to have that new list applied.

ah, ok... that should work. I will look at if there is a post build step where I can put this validation and fail the fail the build in case there are one or more unused --build-args.

• Regarding handling of extraneous --build-arg, I agree with @cpuguy83 here Support for passing build-time variables in build context #15182 (comment): we should ignore or reject them, but not silently accept them.
• I'm not sure I see the point of making the default set of build arguments configurable by the admin as long as we keep this default set general purpose and harmless (which I believe HTTP_PROXY and such absolutely are). What would be the incentive to restrict this list and make it impossible to specify a proxy?

re: configurable - its about who knows what's best. I'm very uncomfortable with the assumption that we, right now, know what the valid list of default ARGs should be for all installs. We've focused on proxy type of stuff, but perhaps some installs have other env vars (or other variants of proxy env vars) that they want to allow w/o having to modify every single image or Dockerfile. As I said, I'm ok with us having a predefined list but let's not make it set in stone such that it can't be changed w/o a PR.

@duglin

I agree with '1' but I save built-in args to the image so that they show-up and are documented in a way identical to author defined args. This saves some special handling for these other than just adding them to list of trusted/allowed args when the build starts. Note that the Dockerfile author may decide to override the default values and description of built-in args in which case it will be desirable to document them.

Having said above, I am ok not saving them in image metadata if they are not overridden by the author. Let me know.

2 - the list of default build-args should be configurable based on the install/admin.

This does provides more flexibility but we will still need some sort of builtin list when a list is not specified by admin (sort of default for a default :)). I think starting off with just a built-in list might be ok for now and we can add this over this change as a separate PR if needed?

3 - related to # 2, over time the list of default args that Docker has may change (grow or shrink) and I wouldn't want to force everyone to have to rebuild all of their images just to have that new list applied.

This is somewhat taken care of in case the list grows here. If an image (img1) was built with say an old built-in list of args. And now someone wants to build a new image using img1 as base and new builder code that has additional built-in args then we add the additional args without needing to build img1.

But I don't remove stuff if the list shrinks as right now the args are inherited from parent image and this somewhat goes back to no special handling for built-in args argument above i.e. if parent image brings in args (including built-in ones) then we don't remove them, which is ok imo as consider parent image might be doing something with those built-in args (like ONBUILD steps?).

A warning could suffice, yes? I don't recall a build tool failing purposefully because of extra environmental values.

I assume this feature will not be available until Docker and Toolbox 1.9+ are released?

@bixu it will be generally available then, but release candidates for 1.9 can be found here: #17000 (comment)

I'll have a look if I can make the time. Is there an approximate ETA for 1.9 GA?

@bixu yup, October 29th is our target. (see https://github.com/docker/docker/milestones).

I also asked for Docker toolbox release-candidates, and that's being worked on, pending release candidates for the other projects that are included in toolbox.

Please consider using Warningf instead of Errorf when len(leftoverArgs) > 0 in evaluator.go.

Something like "Ignoring --build-args [foo,bar,baz] that were not used in dockerfile ARG statements."

The only true call for an error here was

I'd prefer to error out, I think the point of the design is to make sure args are specified in the Dockerfile. ~ @cpuguy83

Respectfully; nothing in the design requires consumption in any the ENV, COPY, ADD or RUN statements. The build-args are explicitly ignored unless explicitly expanded with ARG, right? That silent ignore that @maaquib originally implemented seems to be partially what prompted

right now if someone specifies a "build-arg" that isn't also named via a "ARG" then it is ignored. We should either flag it as an error so the user knows they specified useless info, or we should use it anyway. I'd prefer to use it. ~ @duglin

Its a single word change, yes? Is there a usability or security concern regarding passing --build-arg when there is no corresponding ARG beyond the user wondering why something didn't get used?

I see this issue has been rsolved for quite some time, but in the off chance someone is still monitoring the comment thread I'd like to know if anyone ever got around to creating an issue or pull request to add a feature for customizing the default list of build args supported by Docker. There are several comments in the thread above that suggests doing so, but I couldn't find any links to work subsequently done to move this forward.

If anyone has any links or references they can share I'd appreciate it.

@TheFriendlyCoder the list of build-arg that are currently allowed by default can be found here; /builder/dockerfile/builder.go#L40-L50. If there's particular build-args you are missing, please open a new issue (with a description of your use case)

@thaJeztah
Thanks for the quick response. I assume based on your comment there is currently no way to customize the default list of build arguments other than by updating the "master list" within the Docker source code, correct?

If so, do you know if anyone has created an issue / request to make this list customizable?

There's no such issue/request currently, so feel free to open one. We should properly look into it from a "portability" perspective though, as changing the defaults means that a Dockerfile can only be built if those defaults are set on a daemon.

Hey guys! Is there a way to pass .env-file variable from docker-compose up (or docker-compose build) invocation into the Dockerfile execution environment without really weird hacks? I need to pass USER NAME variable that will be used in build-time environment to become this user. I've tried to do this today and spent over 5 hours with different approaches and not a single one allows me to use the pre-made .env file user variable. It works for me only when I hardcode user name into Dockerfile, sadly. Help would be much appreciated! Thanks...

Please keep in mind that the GitHub issue tracker is not intended as a general support forum,
but for reporting bugs and feature requests. For other type of questions, consider using one of;

• the Docker community Slack channel
• the Docker Support Forums - https://forums.docker.com
• the docker tags on StackOverflow

Let me provide a short example; Also see the documentation at https://docs.docker.com/compose/compose-file/compose-file-v3/#args

(I'm using the v2 (go implementation) of docker compose (compose v2, but this works with docker-compose as well);

Set up the "project";

mkdir dotenv_for_buildargs && cd dotenv_for_buildargs

Compose file:

version: "3.9"
services:
    foo:
        build:
            context: .
            args:
                # no value set here, which makes it take its value from
                # the current environment, or `.env`
                - HELLO

Dockerfile:

FROM alpine
ARG HELLO="default value"
RUN echo "hello $HELLO" > /hello.txt
CMD cat /hello.txt

Now building the image will pick the value for the $HELLO build-arg from either the current environment, or the .env file (if present);

If no .env file is present, and no $HELLO env-var exists in the current environment, the default value as set in the Dockerfile is used:

docker compose build
[+] Building 0.7s (6/6) FINISHED
 => [internal] load build definition from Dockerfile                                          0.0s
 => => transferring dockerfile: 121B                                                          0.0s
 => [internal] load .dockerignore                                                             0.0s
 => => transferring context: 2B                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                              0.0s
 => CACHED [1/2] FROM docker.io/library/alpine                                                0.0s
 => [2/2] RUN echo "hello default value" > /hello.txt                                         0.3s
 => exporting to image                                                                        0.1s
 => => exporting layers                                                                       0.1s
 => => writing image sha256:aafe2b2b43d89608ef49353b1afffbf9e0943e63cc95163bd9bc60dbcc4d20d2  0.0s
 => => naming to docker.io/library/dotenv_for_buildargs_foo                                   0.0s

docker run --rm dotenv_for_buildargs_foo
hello default value

And when using a .env file:

echo "HELLO=world" > ./.env

docker compose build
[+] Building 0.7s (6/6) FINISHED
...
 => [2/2] RUN echo "hello world" > /hello.txt                                                 0.4s
...

docker run --rm dotenv_for_buildargs_foo
hello world

And when setting an environment variable (takes precedence over both .env and the default value):

export HELLO="from environment"

[+] Building 0.5s (6/6) FINISHED
...
 => [2/2] RUN echo "hello from environment" > /hello.txt                                      0.3s
...

docker run --rm dotenv_for_buildargs_foo
hello from environment