chapter1: about SP register

[cch123]

The Go compiler never generates instructions from the PUSH/POP family: the stack is grown or shrunk by respectively decrementing or incrementing the virtual stack pointer SP.

The asm code in this chapter is generated by the compiler, I think Go compiler(1.10) will not generate code that refer to virtual register SP, it should be the hardware SP.

[teh-cmc]

The Go compiler never generates instructions from the PUSH/POP family: the stack is grown or shrunk by respectively decrementing or incrementing the virtual stack pointer SP.

The asm code in this chapter is generated by the compiler, I think Go compiler(1.10) will not generate code that refer to virtual register SP, it should be the hardware SP.

You are definitely right that the virtual stack-pointer has nothing to do with stack-growth, it just wouldn't make any sense.
I wrote this back when I was confused about those virtual registers. I'll fix that in a minute.

On another note, your comment about go not generating code that refer to the virtual SP brought back some dark demons (see #2) of mine, and so I've decided to elucidate this mystery once and for all. Right here, right now.
So, I've been doing some digging and I think you're right, the gc compiler standard debugging tools (e.g. compile -S, tool objdump...) for Go 1.10+ (at least) doesn't don't fiddle around with the virtual stack-pointer at all, despite the documentation firmly stating otherwise. Or at least that's part of the story.

---

I think I'm starting to have a somewhat clear picture of all the intricacies surrounding the virtual stack-pointer (as well as the other virtual registers, really), so I'll try to relay them as best as I can.

$ go version
go version go1.10.1 linux/amd64

For starters, the official documentation states the following:

The SP pseudo-register is a virtual stack pointer used to refer to frame-local variables and the arguments being prepared for function calls. It points to the top of the local stack frame, so references should use negative offsets in the range [−framesize, 0): x-8(SP), y-4(SP), and so on.

It goes on to add this:

On architectures with a hardware register named SP, the name prefix distinguishes references to the virtual stack pointer from references to the architectural SP register. That is, x-8(SP) and -8(SP) are different memory locations: the first refers to the virtual stack pointer pseudo-register, while the second refers to the hardware's SP register.

Now consider this simple code:

func add(a, b int) int { return a + b }

which compiles down to this for linux/amd64:

;; GOOS=linux GOARCH=amd64 go tool compile -S main.go
0x0000 00000 TEXT "".add(SB), NOSPLIT, $0-24
  0x0000 00000 FUNCDATA	$0, gclocals·54241e171da8af6ae173d69da0236748(SB)
  0x0000 00000 FUNCDATA	$1, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
  0x0000 00000 MOVQ	"".b+16(SP), AX   ;; ??
  0x0005 00005 MOVQ	"".a+8(SP), CX    ;; ??
  0x000a 00010 ADDQ	CX, AX
  0x000d 00013 MOVQ	AX, "".~r2+24(SP) ;; ??
  0x0012 00018 RET

Particularly of interest are those three lines:

0x0000 00000 MOVQ	"".b+16(SP), AX   ;; ??
0x0005 00005 MOVQ	"".a+8(SP), CX    ;; ??
  0x000d 00013 MOVQ	AX, "".~r2+24(SP) ;; ??

These three references to the stack-pointer clearly use a named prefix, which should mean that they refer to the virtual stack-pointer, according to the documentation I've just quoted above.
At the same time, though, they use positive offsets to refer to the positions of a and b within the stack-frame; which should be impossible, since the documentation says that the virtual stack-pointer always points to the top of the local frame!

So which is it? What are we really looking at here? The hardware SP or the virtual SP?
Well if we look at the final, native assembly code:

000000000044c150 <main.add>:
  44c150:	48 8b 44 24 10       	mov    0x10(%rsp),%rax
  44c155:	48 8b 4c 24 08       	mov    0x8(%rsp),%rcx
  44c15a:	48 01 c8             	add    %rcx,%rax
  44c15d:	48 89 44 24 18       	mov    %rax,0x18(%rsp)
  44c162:	c3                   	retq   

We can see that all of these three lines were clearly referencing the hardware SP, not the virtual one (i.e. there offsets are not readjusted in any way).
That's because there is no such thing as a virtual SP. It just doesn't exist.
That's because the standard tools (..on amd64!) never reference virtual registers in their assembly outputs. Virtual registers simply don't exist in these outputs.

This whole ordeal is in the same vein as #2; and I'm starting to understand the pattern here.
Most of the documentation surrounding Go's assembly is painfully outdated, and the information in there is now wrong for at least amd64.

Why do I specify "at least"? Because it looks like virtual registers are handled widely differently on each platform (try compiling for arm64 for example), which I'm guessing is the result of having different maintainers and different development speeds on each platform.
Hence while the documentation might still have some correct bits in it for some platforms, it will be outright wrong for others (including amd64).

TL;DR: There is no such thing as a virtual SP or a virtual FP, at least on amd64. Don't get bitten by this.
TL;DR: The standard go tools (..on amd64!) never reference virtual registers in their assembly outputs. Don't look for them. Don't try to make sense of references to SP with named prefixes!

[teh-cmc]

I've linked to this discussion in the relevant part of chapter 1.

I fell like quite a few low-level details of this chapter are gonna need a complete rewrite one of these days, though.

[cch123]

TL;DR; The asm doc is not outdated. The rules in golang.org/doc/asm are only for hand-written asm code, not for generated code.

I found this discussion:

https://groups.google.com/forum/#!searchin/golang-nuts/register$20sp%7Csort:date/golang-nuts/U2kJ7nDTw-s/0GDMGnT2BQAJ

Although they were talking about some other things, but as Rob Pike said, the rules in golang.org/doc/asm are made for hand-written asm code, not for the compiler generated(go tool compile -S or go tool objdump) asm code. The asm code pieces in your project are generated by the compiler, so the SP register is always hardware one(at least for go 1.10).

After the code being compiled, the use of virtual registers will be translated to platform-dependent hardware registers and offsets. If you use virtual SP/FP, it will be translated to hardware SP(with offset adjusted). Sometimes compiler may insert some data(caller BP) to your stack frame, it also needs to adjust the framesize to the correct size. And this insertion behavior also affects the offset of function input param from the hardware SP. (This can be proved by my experiment code in the end of this issue.)

I also did some experiments, and found out that the virtual SP and hardware SP point to the same location when your stack frame's size is 0( when 1. no local variables; 2.no arguments and returns when calling other functions).

They points to different locations when stack frame size is bigger than 0, that is

1. when your framesize is 0($0-xxx):

                                    -----------------
                                     return address
               hardware SP ------> -----------------  <-----virtual SP 

this is the situation for the function in chapter 1 example:

func add(a, b int) int { return a + b }

2. when your framesize is bigger than 0($8-xxx):

                                    -----------------
                                     return address
                                    -----------------
                                        caller BP
                                    -----------------  <-----virtual SP 
                                       local var0
                                    ----------------- <-----hardware SP

Whether caller BP will be inserted to the callee's stack frame needs be decided by two conditions:

1. the stack frame is bigger than 0
2. the following function in go's source code returns true:

func Framepointer_enabled(goos, goarch string) bool {
    return framepointer_enabled != 0 && goarch == "amd64" && goos != "nacl"
}

Also notice that, the name framepointer in golang.org/doc/asm refers to the FP(virtual register), and the framepointer in this Framepointer_enabled function refers to the caller BP(maybe)? They are different....

These conclusions are not explained in go's documents or src code clearly, so it's easy to misunderstand...

ps: my experiment code:

t.go

package main

import (
	"fmt"
)

func output(int) (int, int, int)
func output2(int) (int, int, int)

func main() {
	a, b, c := output(987654321)
	fmt.Println(a, b, c)
	a, b, c = output2(98765)
	fmt.Println(a, b, c)
}

t.s:

// just experiments
// should not use positive offset to virtual SP in your production code

#include "textflag.h"

// func output(int) (int, int)
// frame size is 0 or 8 will affect the result
TEXT ·output(SB), $8-32
	MOVQ 24(SP), DX  // hardware SP + 24(framesize+caller BP+ret address) should points to the first input argument
	MOVQ DX, ret3+24(FP)
	MOVQ argx+16(SP), BX // virtual SP + 16(caller BP+ret address) should points to the first input argument
	MOVQ BX, ret2+16(FP)
	MOVQ arg0+0(FP), AX // first argument
	MOVQ AX, ret1+8(FP)
	RET

TEXT ·output2(SB), $0-32
	MOVQ 8(SP), DX    // hardware SP+8(ret address) points to the first input argument
	MOVQ DX, ret3+24(FP)
	MOVQ argx+8(SP), BX // virtual SP+8(ret address) points to the first input argument
	MOVQ BX, ret2+16(FP)
	MOVQ arg0+0(FP), AX // first argument
	MOVQ AX, ret1+8(FP)
	RET

Put them in same dir, go build and run~

[teh-cmc]

Thanks for the extra digging @cch123, it helps a lot.

TL;DR; The asm doc is not outdated. The rules in golang.org/doc/asm are only for hand-written asm code, not for generated code.

I found this discussion:

https://groups.google.com/forum/#!searchin/golang-nuts/register$20sp%7Csort:date/golang-nuts/U2kJ7nDTw-s/0GDMGnT2BQAJ

Although they were talking about some other things, but as Rob Pike said, the rules in golang.org/doc/asm are made for hand-written asm code, not for the compiler generated(go tool compile -S or go tool objdump) asm code. The asm code pieces in your project are generated by the compiler, so the SP register is always hardware one(at least for go 1.10).

As far as I'm concerned, stating that the documentation isn't outdated, but rather that it's just that the standard tools don't quite follow it anymore is misleading at best...
As you said, nowhere is it mentioned that the documentation only refers to hand-written code. And in fact it does not always follow this rule: some target platforms still do make use of virtual registers in their assembly outputs to this day.

For example, the following code:

func add(a, b int) int { return a + b }

when compiled down to linux/arm64:

0x0000 00000 TEXT "".add(SB), LEAF, $-8-24
  0x0000 00000 MOVD	16(g), R1
  0x0004 00004 MOVD	RSP, R2
  0x0008 00008 CMP	R1, R2
  0x000c 00012 BLS	36
  0x0010 00016 FUNCDATA	ZR, gclocals·54241e171da8af6ae173d69da0236748(SB)
  0x0010 00016 FUNCDATA	$1, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
  0x0010 00016 MOVD	"".a(FP), R0
  0x0014 00020 MOVD	"".b+8(FP), R1
  0x0018 00024 ADD	R1, R0, R0
  0x001c 00028 MOVD	R0, "".~r2+16(FP)
  0x0020 00032 RET	(R30)
  0x0024 00036 NOP
  0x0024 00036 PCDATA	$0, $-1
  0x0024 00036 MOVD	R30, R3
  0x0028 00040 CALL	runtime.morestack_noctxt(SB)
  0x002c 00044 JMP	0

still clearly uses the virtual frame-pointer in its assembly output; even though Keith Randall mentions that "The FP pseudoreg has been dropped from assembly output" in the discussion that you've linked.

I get your point though, and you're right. And this discussion that you've linked to clearly sheds a lot of light on all this confusion.

---

Particularly troubling is this excerpt from the documentation, which has been one of the heaviest point of contention for me:

On architectures with a hardware register named SP, the name prefix distinguishes references to the virtual stack pointer from references to the architectural SP register.

Clearly the assembly output outright ignores this convention, as can be demonstrated in the following code:

0x0000 00000 TEXT "".add(SB), NOSPLIT, $16-24
  0x0000 00000 SUBQ	$16, SP
  0x0004 00004 MOVQ	BP, 8(SP)
  0x0009 00009 LEAQ	8(SP), BP
  0x000e 00014 FUNCDATA	$0, gclocals·54241e171da8af6ae173d69da0236748(SB)
  0x000e 00014 FUNCDATA	$1, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
  0x000e 00014 MOVQ	$10, ""..autotmp_4(SP)
  0x0016 00022 MOVQ	$10, "".y(SB)
  0x0021 00033 MOVQ	""..autotmp_4(SP), AX
  0x0025 00037 MOVQ	AX, "".~r2+40(SP)
  0x002a 00042 MOVQ	8(SP), BP
  0x002f 00047 ADDQ	$16, SP
  0x0033 00051 RET

Here 8(SP) (no prefix) and "".~r2+40(SP) (name-prefix) both refer to the hardware SP, as can be seen by looking at their respective offsets in the final platform-specific assembly, even with a 16-byte local frame:

44c150:	48 83 ec 10          	sub    $0x10,%rsp
44c154:	48 89 6c 24 08       	mov    %rbp,0x8(%rsp)
44c159:	48 8d 6c 24 08       	lea    0x8(%rsp),%rbp     ;; 8(SP), no adjustments
44c15e:	48 c7 04 24 0a 00 00 	movq   $0xa,(%rsp)
44c165:	00
44c166:	48 c7 05 97 67 07 00 	movq   $0xa,0x76797(%rip)
44c16d:	0a 00 00 00
44c171:	48 8b 04 24          	mov    (%rsp),%rax
44c175:	48 89 44 24 28       	mov    %rax,0x28(%rsp)    ;; "".~r2+40(SP), no adjustments
44c17a:	48 8b 6c 24 08       	mov    0x8(%rsp),%rbp
44c17f:	48 83 c4 10          	add    $0x10,%rsp

That's dangerously misleading, imho.

Of course, now that we know that the standard tools never refer to the virtual SP on amd64, this code does make sense. But how would you know that?

---

Your experiments show without a doubt that those virtual registers are still supported by the compiler and work as expected, which I'm not surprised, otherwise we'd have a lot of broken code on our hands.
As you said though, on amd64, the only time we'll see them used is in hand written code, as the standard debugging tools don't care for them.

When I said that these registers don't exist, that's what I meant: that they are never used by the standard tools (-S, objdump...) anymore (for amd64, at least) and that one shouldn't waste his time trying to look for them or trying to make sense of prefixed vs. non-prefixed references to SP, like I did.
I should have made that clearer, you're right. I'll update my comment.

It seems like the rules regarding which platforms use these virtual registers or not in their outputs are purely arbitrary and, AFAIK, undocumented.

Anyway, I've got a much clearer picture of the whole situation now, thanks to you. Kudos.

[teh-cmc]

(I've edited my earlier comment.)

[akutz]

Thank you!! This has been driving me nuts! I just posted about it in Gopher Slack https://gophers.slack.com/archives/C029RQSEE/p1644720012635449.

I'm trying to get the diagrams for https://github.com/akutz/go-interface-values/blob/main/docs/02-interface-values/09-on-the-stack.md correct, and I've been stuck on this issue for days!!

Thank you again!

[jsonlee0x01]

腾讯公司友情提醒：机主暂时奔出服务区、、、呵呵