Help: Converting everything on the G stack to Go.
3,759 views
Skip to first unread message
Keith Randall
Aug 20, 2014, 3:01:24 AM8/20/14
to golang-dev
For 1.4 we're working on converting a bunch of the runtime code from C to Go. The goal is to have exact stack maps for every stack frame visible to the garbage collector or the stack copier. This requirement means that everything on the G stack must be written in Go, as only the Go compiler knows how to generate stack maps. The notable exceptions from this requirement are functions that run on the G stack but can't be seen by the collector/copier (for example, leaf nosplit functions), and assembly functions which have hand-coded stack maps (for example, the reflect.callXX functions.)
I've been working over the summer on a lot of the more complicated rewrites, for instance hash maps (CL 99380043), string operations (CL 93380044), interface conversions (CL 98510044), and so on. We're nearing the finishing line where we have a lot of less complicated conversions still to do, and not much time left to do them. That's where you come in. I've made a list of the remaining work and I am asking for your help to complete it. Go ahead and pick a chunk of whatever size you think you can handle, claim it on the spreadsheet, and dig in.
spreadsheet: https://docs.google.com/spreadsheets/d/1dQ8_hcNYYOKs2BszXUWvxnplsrTwNheVxgzr4y01ogw/edit?usp=sharing
Most of the conversions required are in .goc files. Hopefully at the end of this process we will have only a single .goc file left, stubs.goc, which will contain only recursively nosplit functions. There are a few other conversions which are not in .goc files. I've listed those separately.
Removing a C function from the G stack can be done in a few ways:
- Convert the entire function to Go. Make a .go file corresponding to the .goc file and move the implementation there.
- Convert part of it, calling out to C on the M stack using runtime/stubs.go:onM to do the rest of the work in C. There are examples of how to use onM in runtime/malloc.go and elsewhere. Put the C code that runs on the M stack in a .c file.
- Leave the function in C, but arrange to have it and all of its callees, recursively, labeled as nosplit.
Go definitions of runtime data structures are in the generated file runtime/zruntime_defs_GOOS_GOARCH.go. You'll want to use these definitions in your Go code.
You do not need to convert any nosplit leaf functions. For example, runtime/stubs.goc:acquirem. Also, nosplit functions which call only other nosplit functions, and recursively, are also ok. An example of this is runtime/stubs.goc:gocas, as it only calls cas which is itself a nosplit leaf function. If you find functions that require no changes, just mark them as done on the spreadsheet.
This farming out of work is an experiment of a sort, so I'm happy to answer any questions you have. Also, if you notice any functions you think need converting but are not covered by the spreadsheet, let me know.
-Keith
Dave Cheney
Aug 20, 2014, 4:51:21 AM8/20/14
to Keith Randall, golang-dev
Thanks Keith.
I have a memory that there is a mode we can put the runtime into where it will log or panic if C code ends up on the Go stack. Is that true? Is such a thing possible ?
Dave
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Daniel Morsing
Aug 20, 2014, 7:11:07 AM8/20/14
to golan...@googlegroups.com
Thanks Keith.
What's the plan for C "..." args in deferproc and newproc? Switching to the M stack with a pointer to the args might not work since the stack can move from underneath the C code.
Regards,
Daniel Morsing
Dmitry Vyukov
Aug 20, 2014, 7:24:50 AM8/20/14
to Daniel Morsing, golang-dev
on g0 without possibility of rescheduling/gcing/stackshrinking in
between. So you still have a valid args pointer into g stack.
ericdr...@gmail.com
Aug 20, 2014, 8:54:56 AM8/20/14
to golan...@googlegroups.com
Hi Keith:
I'm trying to write more and more code in Go and would like to help. I'm concerned that I don't have the background for this task however.
I have a fairly good background in C and have been writing Go as much as possible over the last 12 to 15 months.
However, I have no idea what a G stack or a M stack is and it doesn't appear that there's a lot of material beyond the code to get one up to speed.
Is there too much of a learning curve to take on something like this and contribute before it's done.
I understand this is somewhat subjective. I'm just looking for some guidance.
Thanks
Eric
Dmitry Vyukov
Aug 20, 2014, 10:07:42 AM8/20/14
to Keith Randall, golang-dev
Is there anything special about calling Go code on g0 stack long-term?
Dmitry Vyukov
Aug 20, 2014, 12:36:16 PM8/20/14
to Keith Randall, golang-dev
the table is missing functions from proc.c - newproc, ready
some of them are in miscellaneous, but that's also significant chunk of work
some of them are in miscellaneous, but that's also significant chunk of work
Keith Randall
Aug 20, 2014, 1:31:58 PM8/20/14
to Dmitry Vyukov, golang-dev
> I'd like to take mprof.goc, i'm not able to edit the spreadsheet right now.
>
> Rémy.
>
> Rémy.
Rémy, I added you.
> I have a memory that there is a mode we can put the runtime into where it will log or panic if C > code ends up on the Go stack. Is that true? Is such a thing possible ?
No such mode currently exists. I have a CL which does something like this, in the linker record (and eventually, forbid) when Go code calls C code. There are a few exceptions that need to be hardcoded, like mcall and newstack. If we get this all cleaned up for 1.4 we can enable it to prevent regression.
> Is there anything special about calling Go code on g0 stack long-term?
I have a vision of a strict separation, Go code only on the G stack and C code only on the M stack (and a few assembly routines, like memmove, allowed on both). But that's harder and won't happen for 1.4. In particular the Go malloc code runs on the M stack right now.
While there is Go code on the M stack, we do have to ensure that that Go code uses bounded stack.
> However, I have no idea what a G stack or a M stack is and it doesn't appear that there's a lot > of material beyond the code to get one up to speed.
>
> Is there too much of a learning curve to take on something like this and contribute before it's done.
>
> I understand this is somewhat subjective. I'm just looking for some guidance.
Unfortunately, this task has a steep learning curve for people who haven't worked on the runtime before.
brainman
Aug 20, 2014, 8:48:23 PM8/20/14
to golan...@googlegroups.com
I am trying to convert syscall_windows.goc. The file belongs to "syscall" package, not "runtime". I can move that file to "syscall", but it uses some internal runtime types, variables and functions. How do you propose I approach that? asm?
Thank you.
ALex
minux
Aug 21, 2014, 12:35:43 AM8/21/14
to Alex Brainman, golang-dev
On Aug 20, 2014 8:48 PM, "brainman" <alex.b...@gmail.com> wrote:
> I am trying to convert syscall_windows.goc. The file belongs to "syscall" package, not "runtime". I can move that file to "syscall", but it uses some internal runtime types, variables and functions. How do you propose I approach that? asm?
i think the only way to go is to move it to syscall, write in Go that calls assembly forwarding function (just jmp, don't use multiple files for each arch., use #ifdef GOARCH_arm and #define JMP B) to the runtime internal functions.
brainman
Aug 21, 2014, 12:38:43 AM8/21/14
to golan...@googlegroups.com, alex.b...@gmail.com
> ... use #ifdef GOARCH_arm and #define JMP B)
care to point me to an example file?
Alex
minux
Aug 21, 2014, 1:12:59 AM8/21/14
to Alex Brainman, golang-dev
:
On Aug 21, 2014 12:38 AM, "brainman" <alex.b...@gmail.com> wrote:
> > ... use #ifdef GOARCH_arm and #define JMP B)
> care to point me to an example file?
i just realized that we don't have windows/arm support atm, but anyway, here is one example:
http://tip.golang.org/src/pkg/os/signal/sig.s
it builds on all three supported architectures (not power64 though, because it uses BR, but don't worry about power64 now)
brainman
Aug 21, 2014, 1:27:30 AM8/21/14
to golan...@googlegroups.com, alex.b...@gmail.com
On Thursday, 21 August 2014 15:12:59 UTC+10, minux wrote:
> ... here is one example:
http://tip.golang.org/src/pkg/os/signal/sig.s
Thank you.
> ... here is one example:
http://tip.golang.org/src/pkg/os/signal/sig.s
Thank you.
Alex
sanjay.m
Aug 21, 2014, 1:29:08 AM8/21/14
to golan...@googlegroups.com, alex.b...@gmail.com
Thanks for the example; I am having the same problem with rdebug.goc (it declares symbols for runtime/debug, and uses runtime internal functions).
To clarify, if the runtime-internal function is a Go function and the function take an argument, does the assembly stub still say "NOSPLIT,$0"?
I ask because the NOSPLIT docs seem to say that the NOSPLIT-ted function's frame needs space for all called functions, but I feel like that isn't true if a called function does the stack split check in its own preamble.
Thanks,
Sanjay
Dmitry Vyukov
Aug 21, 2014, 1:50:02 AM8/21/14
to minux, Alex Brainman, golang-dev
thunks to call unexported runtime functions from syscall package.
I have not looked at that particular case, but I think that
syscall<->runtime interface is smaller and better defined and does not
use runtime-internal types.
Dmitry Vyukov
Aug 21, 2014, 1:51:22 AM8/21/14
to sanjay.m, golang-dev, Alex Brainman
Dmitry Vyukov
Aug 21, 2014, 1:52:30 AM8/21/14
to minux, Alex Brainman, golang-dev
Alex, example of such approach is in
https://codereview.appspot.com/127460044/ in reflect asm files.
Alternatively, runtime asm files still can be used to "implement"
functions in syscall package.
https://codereview.appspot.com/127460044/ in reflect asm files.
Alternatively, runtime asm files still can be used to "implement"
functions in syscall package.
brainman
Aug 21, 2014, 1:56:11 AM8/21/14
to golan...@googlegroups.com, mi...@golang.org, alex.b...@gmail.com
On Thursday, 21 August 2014 15:50:02 UTC+10, Dmitry Vyukov wrote:
> ... The better options is to keep that file in runtime, and use the asm
> ... The better options is to keep that file in runtime, and use the asm
> thunks to call unexported runtime functions from syscall package.
That is what I am doing now. Thank you.
Alex
minux
Aug 21, 2014, 10:22:31 AM8/21/14
to Sanjay Menakuru, Alex Brainman, golang-dev
On Aug 21, 2014 1:29 AM, "sanjay.m" <balas...@gmail.com> wrote:
> Thanks for the example; I am having the same problem with rdebug.goc (it declares symbols for runtime/debug, and uses runtime internal functions).
>
> To clarify, if the runtime-internal function is a Go function and the function take an argument, does the assembly stub still say "NOSPLIT,$0"?
yes.
> I ask because the NOSPLIT docs seem to say that the NOSPLIT-ted function's frame needs space for all called functions, but I feel like that isn't true if a called function does the stack split check in its own preamble.
it's ok for nosplit function to split stack function.
Rob Pike
Aug 21, 2014, 10:25:33 AM8/21/14
to minux, Sanjay Menakuru, Alex Brainman, golang-dev
I think minux meant to write
It's OK for a NOSPLIT function to call a function that can trigger
a stack split.
The NOSPLIT marker refers only to the current frame.
By the way, a NOSPLIT function doesn't always have $0 as the last
element of its TEXT directive. It can't have locals (or the stack
would split) but it can have arguments, and the $X value should report
the size of the argument list.
-rob
It's OK for a NOSPLIT function to call a function that can trigger
a stack split.
The NOSPLIT marker refers only to the current frame.
By the way, a NOSPLIT function doesn't always have $0 as the last
element of its TEXT directive. It can't have locals (or the stack
would split) but it can have arguments, and the $X value should report
the size of the argument list.
-rob
Dmitry Vyukov
Aug 21, 2014, 10:29:09 AM8/21/14
to Rob Pike, minux, Sanjay Menakuru, Alex Brainman, golang-dev
On Thu, Aug 21, 2014 at 6:24 PM, Rob Pike <r...@golang.org> wrote:
> I think minux meant to write
> It's OK for a NOSPLIT function to call a function that can trigger
> a stack split.
> The NOSPLIT marker refers only to the current frame.
>
> By the way, a NOSPLIT function doesn't always have $0 as the last
> element of its TEXT directive. It can't have locals (or the stack
> would split) but it can have arguments, and the $X value should report
> the size of the argument list.
A NOSPLIT function can have locals, just up to some limit. See e.g.
> I think minux meant to write
> It's OK for a NOSPLIT function to call a function that can trigger
> a stack split.
> The NOSPLIT marker refers only to the current frame.
>
> By the way, a NOSPLIT function doesn't always have $0 as the last
> element of its TEXT directive. It can't have locals (or the stack
> would split) but it can have arguments, and the $X value should report
> the size of the argument list.
entersyscallblock in proc.c, it's NOSPLIT and has local vars.
Dmitry Vyukov
Aug 21, 2014, 10:44:45 AM8/21/14
to Keith Randall, Russ Cox, golang-dev
On Wed, Aug 20, 2014 at 9:31 PM, Keith Randall <k...@google.com> wrote:
>> Is there anything special about calling Go code on g0 stack long-term?
>
> I have a vision of a strict separation, Go code only on the G stack and C
> code only on the M stack (and a few assembly routines, like memmove, allowed
> on both). But that's harder and won't happen for 1.4. In particular the Go
> malloc code runs on the M stack right now.
>
> While there is Go code on the M stack, we do have to ensure that that Go
> code uses bounded stack.
Humm... I though that we want to get rid of C compiler altogether.
>> Is there anything special about calling Go code on g0 stack long-term?
>
> I have a vision of a strict separation, Go code only on the G stack and C
> code only on the M stack (and a few assembly routines, like memmove, allowed
> on both). But that's harder and won't happen for 1.4. In particular the Go
> malloc code runs on the M stack right now.
>
> While there is Go code on the M stack, we do have to ensure that that Go
> code uses bounded stack.
Then it does not need to be maintained, converted to Go and optimized.
I am pretty sure there is still a bunch codegen bugs. Instead of
giving me compilation errors, it silently crashes. Quality of code
that it generates is worse than Go already, and nobody is going to
improve it.
What are the reasons to have C code long-term?
If we want to do some optimizations based on g/g0 or otherwise
distinguish between them, we can have a //go:systemstack annotation
(that works only in runtime package).
Russ?
Dmitry Vyukov
Aug 21, 2014, 10:47:45 AM8/21/14
to Keith Randall, Russ Cox, golang-dev
If I need to duplicate a C function in Go, is a temporal duplication or not?
Is there any value in converting to Go something that is not
performance critical and can perfectly live with onM switch?
minux
Aug 21, 2014, 10:57:10 AM8/21/14
to Dmitry Vyukov, Russ Cox, Keith Randall, golang-dev
On Aug 21, 2014 10:44 AM, "'Dmitry Vyukov' via golang-dev" <golan...@googlegroups.com> wrote:
>
> On Wed, Aug 20, 2014 at 9:31 PM, Keith Randall <k...@google.com> wrote:
>
> >> Is there anything special about calling Go code on g0 stack long-term?
> >
> > I have a vision of a strict separation, Go code only on the G stack and C
> > code only on the M stack (and a few assembly routines, like memmove, allowed
> > on both). But that's harder and won't happen for 1.4. In particular the Go
> > malloc code runs on the M stack right now.
> >
> > While there is Go code on the M stack, we do have to ensure that that Go
> > code uses bounded stack.
>
> Humm... I though that we want to get rid of C compiler altogether.
i believe that's still the plan.
russ suggested that we might be able to use his c2go to convert the remaining c code, but we're not there yet.
Keith Randall
Aug 21, 2014, 11:47:08 AM8/21/14
to minux, Dmitry Vyukov, Russ Cox, golang-dev
Yes, I believe the long term plan is to get rid of C altogether. I wouldn't want to maintain duplicate code until that happens, though.
Ian Lance Taylor
Aug 21, 2014, 1:20:50 PM8/21/14
to Dmitry Vyukov, Keith Randall, Russ Cox, golang-dev
On Thu, Aug 21, 2014 at 7:44 AM, 'Dmitry Vyukov' via golang-dev
<golan...@googlegroups.com> wrote:
>
> Humm... I though that we want to get rid of C compiler altogether.
>
> Then it does not need to be maintained, converted to Go and optimized.
> I am pretty sure there is still a bunch codegen bugs. Instead of
> giving me compilation errors, it silently crashes. Quality of code
> that it generates is worse than Go already, and nobody is going to
> improve it.
>
> What are the reasons to have C code long-term?
> If we want to do some optimizations based on g/g0 or otherwise
> distinguish between them, we can have a //go:systemstack annotation
> (that works only in runtime package).
When all C code lives on the M stack, we could reasonably consider
<golan...@googlegroups.com> wrote:
>
> Humm... I though that we want to get rid of C compiler altogether.
>
> Then it does not need to be maintained, converted to Go and optimized.
> I am pretty sure there is still a bunch codegen bugs. Instead of
> giving me compilation errors, it silently crashes. Quality of code
> that it generates is worse than Go already, and nobody is going to
> improve it.
>
> What are the reasons to have C code long-term?
> If we want to do some optimizations based on g/g0 or otherwise
> distinguish between them, we can have a //go:systemstack annotation
> (that works only in runtime package).
compiling it with GCC/Clang. I don't know if that is a good path to
follow, but the point is that the goal of removing the C compiler from
the sources is separable from the goal of converting all runtime code
to Go.
Ian
Dmitry Vyukov
Aug 21, 2014, 1:26:18 PM8/21/14
to Ian Lance Taylor, Keith Randall, Russ Cox, golang-dev
From: Russ Cox
Date: Wed, Feb 12, 2014 at 9:50 PM
Subject: Re: CL for growing stacks
To: Dmitry Vyukov
On Wed, Feb 12, 2014 at 12:49 PM, Dmitry Vyukov wrote:
>
> Compiling inner GC loop with gcc/clang would be awesome!
Except for having to build gcc/clang to build Go, and also having to
debug performance differences (or plain bugs) caused by different
people compiling the same Go version with different gcc/clang. Let's
not get distracted.
Russ
zhuowe...@yahoo.com
Aug 24, 2014, 9:58:58 PM8/24/14
to golan...@googlegroups.com
On Wednesday, August 20, 2014 12:01:24 AM UTC-7, Keith Randall wrote:
Removing a C function from the G stack can be done in a few ways:
I noticed that syscall_solaris.goc and syscall_windows.goc both call runtime.cgocall, which needs the currently executing g, so can't be executed on the m stack (using g0) - how should one convert them, then?
Dmitry Vyukov
Aug 25, 2014, 3:21:47 PM8/25/14
to zhuowe...@yahoo.com, golang-dev
brainman
Aug 26, 2014, 11:02:59 PM8/26/14
to golan...@googlegroups.com
I have some questions.
I tried to convert Syscall, but it calls runtime.cgocall, and runtime.cgocall is not converted to Go yet. Converting runtime.cgocall is, probably, too much for me - I am not familiar with that code at all. We would have to convert all cgo related code together.
Also converted Syscall becomes inefficient. It uses LibCall local variable, and I have to use new(libcall) for that, but that calls runtime.newobject. Also current Syscall uses parameters on stack as is by taking address of the first parameter. I could not do that in Go - it says "escapes to heap, not allowed in runtime." The only way to overcome that is to create another local array (another runtime.newobject call) and copy parameters there.
I can, probably, convert Syscall into asm, but it would still have to call runtime.cgocall, therefore it will not be NOSPLIT. Should I do that? I see no benefit.
Looking for suggestions. Thank you.
Alex
Dmitry Vyukov
Aug 27, 2014, 5:41:03 AM8/27/14
to brainman, golang-dev
On Wed, Aug 27, 2014 at 7:02 AM, brainman <alex.b...@gmail.com> wrote:
> I have some questions.
>
> I tried to convert Syscall, but it calls runtime.cgocall, and
> runtime.cgocall is not converted to Go yet. Converting runtime.cgocall is,
> probably, too much for me - I am not familiar with that code at all. We
> would have to convert all cgo related code together.
Start with just Syscall, we will convert cgocall later.
> I have some questions.
>
> I tried to convert Syscall, but it calls runtime.cgocall, and
> runtime.cgocall is not converted to Go yet. Converting runtime.cgocall is,
> probably, too much for me - I am not familiar with that code at all. We
> would have to convert all cgo related code together.
> Also converted Syscall becomes inefficient. It uses LibCall local variable,
> and I have to use new(libcall) for that, but that calls runtime.newobject.
> Also current Syscall uses parameters on stack as is by taking address of the
> first parameter. I could not do that in Go - it says "escapes to heap, not
> allowed in runtime." The only way to overcome that is to create another
> local array (another runtime.newobject call) and copy parameters there.
>
> I can, probably, convert Syscall into asm, but it would still have to call
> runtime.cgocall, therefore it will not be NOSPLIT. Should I do that? I see
> no benefit.
>
> Looking for suggestions. Thank you.
See example below, I allocate keventstruct on stack and pass its
address to syscall:
https://codereview.appspot.com/132910043/diff/140001/src/pkg/runtime/netpoll_kqueue.go
Aram Hăvărneanu
Aug 27, 2014, 8:33:53 AM8/27/14
to Dmitry Vyukov, brainman, golang-dev
I assigned myself to syscall_solaris.goc.
Why are some solaris functions from other files missing, say functions
from os_solaris.c?
--
Aram Hăvărneanu
Why are some solaris functions from other files missing, say functions
from os_solaris.c?
--
Aram Hăvărneanu
Dmitry Vyukov
Aug 27, 2014, 8:40:18 AM8/27/14
to Aram Hăvărneanu, brainman, golang-dev
On Wed, Aug 27, 2014 at 4:33 PM, Aram Hăvărneanu <ara...@mgk.ro> wrote:
> I assigned myself to syscall_solaris.goc.
>
> Why are some solaris functions from other files missing, say functions
> from os_solaris.c?
I suspect Keith looked at just .goc files. The ultimate criteria -- is
> I assigned myself to syscall_solaris.goc.
>
> Why are some solaris functions from other files missing, say functions
> from os_solaris.c?
it running on g stack? If yes, then it must be converted. If it runs
only on system g0 stack, then it does not need to be converted right
now.
Aram Hăvărneanu
Aug 28, 2014, 11:28:00 AM8/28/14
to Dmitry Vyukov, brainman, golang-dev
Is it possible to call a variadic C function from Go?
--
Aram Hăvărneanu
--
Aram Hăvărneanu
Ian Lance Taylor
Aug 28, 2014, 11:31:15 AM8/28/14
to Aram Hăvărneanu, Dmitry Vyukov, brainman, golang-dev
On Thu, Aug 28, 2014 at 8:27 AM, Aram Hăvărneanu <ara...@mgk.ro> wrote:
>
> Is it possible to call a variadic C function from Go?
It should work fine in the runtime package, except that the Go code
>
> Is it possible to call a variadic C function from Go?
will not be able to see any return value.
Ian
Dmitry Vyukov
Aug 28, 2014, 11:40:02 AM8/28/14
to Aram Hăvărneanu, brainman, golang-dev
If you want to convert solaris syscalls, then you can check out what I
am doing for windows' stdcall:
https://codereview.appspot.com/135090043
It's not yet done completely, and it was not reviewed, but it should work.
am doing for windows' stdcall:
https://codereview.appspot.com/135090043
It's not yet done completely, and it was not reviewed, but it should work.
Reply all
Reply to author
Forward
0 new messages