Remove Unnecessary Memory Copies with Slice to Array Conversions

[patrick-ogrady]

go1.17 added the ability to convert from slice to array/array pointer: https://tip.golang.org/ref/spec#Conversions_from_slice_to_array_or_array_pointer

However, the UX for this was super ugly until go1.20 (just released):

We should now migrate all our codec unmarshaling to do this instead of copying from the tx byte slice (which will dramatically reduce memory allocations in block processing). Thanks to @rafael-abuawad for calling my attention to this in #21.

I wish we had more memory allocations benchmarked so we could show the performance improvement here but don't think it makes sense to wait to add those to add this. We can probably just add one quick one for unmarshaling a dummy tx.

We MUST be careful to only cast exactly the bytes we need so we don't keep the entire backing array of the slice alive if we don't need to (eventhough we usually will because we hold onto the tx bytes): https://utcc.utoronto.ca/~cks/space/blog/programming/GoInteriorPointerGC

[rafael-abuawad]

#21 A optimization that I found was changing this function:

// PublicKey returns a PublicKey associated with the Ed25519 PrivateKey p.
// The PublicKey is the last 32 bytes of p.
func (p PrivateKey) PublicKey() PublicKey {
	rpk := p[32:] // privateKey == private|public
	var pk PublicKey
	copy(pk[:], rpk)
	return pk
}

With this:

// PublicKey returns a PublicKey associated with the Ed25519 PrivateKey p.
// The PublicKey is the last 32 bytes of p.
func (p PrivateKey) PublicKey() PublicKey {
	return PublicKey(p[32:])
}

[patrick-ogrady]

This work is now unblocked!

[patrick-ogrady]

This will likely involve converting a bunch of arrays to pointers to arrays so we can reuse memory from parsed blocks.

[github-actions]

This issue has become stale because it has been open 60 days with no activity. Adding the lifecycle/frozen label will exempt this issue from future lifecycle events.