At this point, I suppose I am as responsible for the code and decisions the author is complaining about as anyone.
Different people come from different backgrounds and, based on that, find different tools useful. Ultimately you need to use the tools that let you get your job done most effectively, however that is defined in your particular case.
The Plan 9 toolchain has worked and continues to work well for Go. Most importantly, it has been something we understand completely and is quite easy to adapt as needed. The standard ABIs and toolchains are for C, and the decisions made there may be completely inappropriate for languages with actual runtimes.
For example, no standard ABIs and toolchains supported segmented stacks; we had to build that, so it was going to be incompatible from day one. If step one had been "learn the GCC or LLVM toolchains well enough to add segmented stacks", I'm not sure we'd have gotten to step two. (Later, for gccgo, Ian did have to add segmented stack support to the GNU gold linker, but Ian also wrote that linker, so he was uniquely prepared.)
As another example, Go 1.4 replaces segmented stacks with stacks that grow or shrink, as needed, by moving. And Go 1.4's garbage collector (GC) is also completely precise wrt stack references into the heap. These require precise information about which words on the stack are pointers and, for the GC, which words are initialized and live. Go 1.5 will use that same information to introduce a concurrent (bounded pause) garbage collector. Getting the information through to the runtime was non-trivial but possible for us to do with the Plan 9 toolchain. If we'd built on GCC, it would have required first threading all that stack map information through the GCC or LLVM backends and into a form that the runtime can use efficiently. Like before, I doubt we'd have gotten to step two. (I don't know how gccgo is going to tackle this. It's a big undertaking. I expect gccgo will have segmented stacks and imprecise stack scans for a while yet.)
The toolchain is quirky and flawed in many ways. Some people don't like typing shift so much in assembly files. Some people don't like the center dot hack (glad the author didn't find the Unicode slash!). Those are really cosmetic details, and if you stop there, you miss the point, which is that it's a small toolchain that we can keep in our heads and make arbitrary changes to, quickly and easily. Honestly, if we'd built on GCC or LLVM, we'd be moving so slowly I'd probably have left the project years ago.
The linker in particular is a piece of code in transition. The author is 100% right that it's not great code. But it does work well, and it is being put to production use at many companies as you read this. The code to produce ELF object files and dynamically linked binaries in particular is new, is my fault, and was quite difficult to fit into the original structure. The linker is overdue to be replaced. The new linker is in the Go tree in src/cmd/link, but it's unfinished.
Using a standard linker instead doesn't work well for a managed language. The Go runtime needs to know where all the pointers are in the data and bss segments, and a standard linker won't put that information together for you. The Go linker does that. The Go linker also writes out runtime information for unwinding stacks during GC and translating program counters to file:line for use in stack dumps. Again, this is all much easier to do with a toolchain that we control entirely than having to build into existing ones. (I am not aware of any way to get file:line information through an existing linker except in DWARF tables, which is overkill. And then there's OS X and Windows, where you _really_ have no control over the toolchain.)
The calls to throw everything away and start over are at best hyperbolic. Certainly there is room for improvement, but the custom toolchain is one of the key reasons we've accomplished so much in so little time. I think the author doesn't fully appreciate all the reasons that C tools don't work well for languages that do so much more than C. Most other managed languages are doing JIT compilation and don't go through standard linkers either.
Thank you for posting that. It was never a mystery why the project started with the Plan 9 C compiler (if you're Ken Thompson, of course you'll use Ken Thompson's compiler) but the detail on the evolution of the thing is interesting.
It is a bit embarrassing that our self-moderation here did not cause your comment to rise a little higher toward the top.
This is an excellent reply, but it makes crystal clear one point: Go doesn't produce "native" code other than in the very limited sense of binaries that can be loaded directly by the OS. Go is a lot more like Java than C or Rust. It is Java with some different decisions than HotSpot: all code is AOT-compiled, and the result is statically linked with the "VM". In fact, because the JVM spec says nothing about JITs or static linking, a JVM that is implemented just like Go is a perfectly compliant JVM, and, in fact, there are some JVMs ([1], [2]) that behave exactly the same way (they're used for hard realtime embedded software in avionics and similar domains)
Different people come from different backgrounds and, based on that, find different tools useful. Ultimately you need to use the tools that let you get your job done most effectively, however that is defined in your particular case.
The Plan 9 toolchain has worked and continues to work well for Go. Most importantly, it has been something we understand completely and is quite easy to adapt as needed. The standard ABIs and toolchains are for C, and the decisions made there may be completely inappropriate for languages with actual runtimes.
For example, no standard ABIs and toolchains supported segmented stacks; we had to build that, so it was going to be incompatible from day one. If step one had been "learn the GCC or LLVM toolchains well enough to add segmented stacks", I'm not sure we'd have gotten to step two. (Later, for gccgo, Ian did have to add segmented stack support to the GNU gold linker, but Ian also wrote that linker, so he was uniquely prepared.)
As another example, Go 1.4 replaces segmented stacks with stacks that grow or shrink, as needed, by moving. And Go 1.4's garbage collector (GC) is also completely precise wrt stack references into the heap. These require precise information about which words on the stack are pointers and, for the GC, which words are initialized and live. Go 1.5 will use that same information to introduce a concurrent (bounded pause) garbage collector. Getting the information through to the runtime was non-trivial but possible for us to do with the Plan 9 toolchain. If we'd built on GCC, it would have required first threading all that stack map information through the GCC or LLVM backends and into a form that the runtime can use efficiently. Like before, I doubt we'd have gotten to step two. (I don't know how gccgo is going to tackle this. It's a big undertaking. I expect gccgo will have segmented stacks and imprecise stack scans for a while yet.)
The toolchain is quirky and flawed in many ways. Some people don't like typing shift so much in assembly files. Some people don't like the center dot hack (glad the author didn't find the Unicode slash!). Those are really cosmetic details, and if you stop there, you miss the point, which is that it's a small toolchain that we can keep in our heads and make arbitrary changes to, quickly and easily. Honestly, if we'd built on GCC or LLVM, we'd be moving so slowly I'd probably have left the project years ago.
The linker in particular is a piece of code in transition. The author is 100% right that it's not great code. But it does work well, and it is being put to production use at many companies as you read this. The code to produce ELF object files and dynamically linked binaries in particular is new, is my fault, and was quite difficult to fit into the original structure. The linker is overdue to be replaced. The new linker is in the Go tree in src/cmd/link, but it's unfinished.
Using a standard linker instead doesn't work well for a managed language. The Go runtime needs to know where all the pointers are in the data and bss segments, and a standard linker won't put that information together for you. The Go linker does that. The Go linker also writes out runtime information for unwinding stacks during GC and translating program counters to file:line for use in stack dumps. Again, this is all much easier to do with a toolchain that we control entirely than having to build into existing ones. (I am not aware of any way to get file:line information through an existing linker except in DWARF tables, which is overkill. And then there's OS X and Windows, where you _really_ have no control over the toolchain.)
The calls to throw everything away and start over are at best hyperbolic. Certainly there is room for improvement, but the custom toolchain is one of the key reasons we've accomplished so much in so little time. I think the author doesn't fully appreciate all the reasons that C tools don't work well for languages that do so much more than C. Most other managed languages are doing JIT compilation and don't go through standard linkers either.