Go currently (pre 1.5) doesn’t provide any built in method for vendoring packages. The tools that currently exist today (godep, nut and a few others) basically exploit the implementation details of $GOPATH. Go 1.5 however includes a “vendor experiment”. With this experiment `go` commands will attempt to resolve dependencies in `vendor/` directories.
Russ Cox explains it fairly well in his commit message:
If there is a source directory d/vendor, then, when compiling a source file within the subtree rooted at d, import "p" is interpreted as import "d/vendor/p" if that exists.
When there are multiple possible resolutions, the most specific (longest) path wins.
The short form must always be used: no import path can contain “/vendor/” explicitly.
Import comments are ignored in vendored packages.To utilize this feature you need to set the environment variable `GO15VENDOREXPERIMENT` to `1` before running any of the go commands (run, test, build, install, etc)
For instance given a source tree like this:
./main.go
./vendor
./vendor/a
./vendor/a/a.go
./vendor/b
./vendor/b/b.go
./vendor/c
./vendor/c/c.go
./vendor/c/vendor
./vendor/c/vendor/a
./vendor/c/vendor/a/a.goIn this example I have main.go requiring all of the packages like so:
package mainimport (
“a”
“b”
“c”
“fmt”
)func main() {
fmt.Println(“main.main()”)
fmt.Println(“main: a.A ==“, a.A)
fmt.Println("main: a.Inc() ==", a.Inc())
fmt.Println(“main: b.B ==“, b.B)
fmt.Println(“main: c.C ==“, c.C)
}
vendor/a/a.go looks like:
package aimport “fmt”var A = 1func Inc() int {
A++
return A
}func init() {
fmt.Println(“Init package a”)
fmt.Println(“a: A ==“, A)
}
vendor/b/b.go looks like:
package bimport “a”
import “fmt”var B = 2func init() {
fmt.Println(“Init package b”)
fmt.Println(“b: a.A ==“, a.A)
fmt.Println(“b: a.Inc() ==“, a.Inc())
}
vendor/c/c.go looks like:
package cimport “a”
import “fmt”var C = 3func init() {
fmt.Println(“Init package c”)
fmt.Println(“c: C ==“, C)
fmt.Println(“c: a.A ==“, a.A)
fmt.Println(“c: a.Inc() ==“, a.Inc())
}
and lastly vendor/c/vendor/a/a.go looks like:
package aimport “fmt”var A = 100func Inc() int {
A++
return A
}func init() {
fmt.Println(“Init c/vendor/a”)
fmt.Println(“c/vendor/a: A ==“, A)
}
Running main.go without the vendor experiment produces what you would expect:
$ go run main.go
main.go:4:2: cannot find package “a” in any of:
/usr/local/go/src/a (from $GOROOT)
/Users/emuller/go/src/a (from $GOPATH)
main.go:5:2: cannot find package “b” in any of:
/usr/local/go/src/b (from $GOROOT)
/Users/emuller/go/src/b (from $GOPATH)
main.go:6:2: cannot find package “c” in any of:
/usr/local/go/src/c (from $GOROOT)
/Users/emuller/go/src/c (from $GOPATH)But running main.go with the vendor experiment turned on produces:
$ GO15VENDOREXPERIMENT=1 go run main.go
Init package a
a: A == 1
Init package b
b: a.A == 1
b: a.Inc() == 2
Init c/vendor/a
c/vendor/a: A == 100
Init package c
c: C == 3
c: a.A == 100
c: a.Inc() == 101
main.main()
main: a.A == 2
main: a.Inc() == 3
main: b.B == 2
main: c.C == 3Let’s unpack that a little….
main.go imports all three top level dependencies: a, b & c.
Each of those dependencies uses an init function for demonstration purposes. As each init function is executed it prints out its exported variable. When it imports package a it also calls a’s exported Inc function so we can observe a side effect.
First we see package a being initialized:
Init package a
a: A == 1Then we see package b being initialized:
Init package b
b: a.A == 1
b: a.Inc() == 2Package b imports package a and because of the dependency resolution policy of longest path wins (see Russ’ comment above) vendor/a is what is used. a.Inc is called, which increments a’s internal variable.
Then we see package c being initialized:
Init c/vendor/a
c/vendor/a: A == 100
Init package c
c: C == 3
c: a.A == 100
c: a.Inc() == 101Package c also imports a package a, but unlike package b it has its own version of a package a inside of its own vendor directory making the full path to the package a that package c imports being: vendor/c/vendor/a. This new package a has a different value for its internal variable (100) and calling a.Inc updates it to 101.
Lastly main’s main function runs:
main.main()
main: a.A == 2
main: a.Inc() == 3
main: b.B == 2
main: c.C == 3Showing that the package a used by main is the one in vendor/a, not the one in vendor/c/vendor/a or vice versa.
Recursion!
One interesting side effect of this experimental implementation is that it’s recursive in nature, meaning libraries with dependencies can vendor their own dependencies inside their own vendor directory and the go tooling handles it.
Side Effects + Gotchas
Multiple versions of the same library will cause problems. For instance, only a single copy of any database/sql driver can be registered, if you vendor lib/pq and another vendored dep also vendors it and imports it, you’ll get a panic. This is a limitation of database/sql, but probably also a sane one. Another issue (hat tip to onetruekarl) comes up when 2 vendored dependencies also vendor their own copies of another depencency with the same name. If any of those sub types bubble out of the libraries that you’ve vendored you won’t be able to tell which vendored package it actually comes from.
Tools like godep, vendor, gb, etc may be called upon to “squash” down recursively vendored packages into a single version in the vendor/ directory. It’s not really clear what people’s expectations here are though.
There is a strong argument to be made that libraries should not vendor their own dependencies, but as the author of several library / command combo packages that rely extensively on 3rd party libraries I’m not convinced of that blanket assertion.
Determining the proper versions without a bunch of extra metadata around will be hard. This is probably a place where the vendor-spec comes in handy.
In any case I am looking forward to the official support for this.
Originally published at icanhazdowntime.org.
