Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing optimisations #2937

Merged
merged 2 commits into from
Nov 16, 2017
Merged

Parsing optimisations #2937

merged 2 commits into from
Nov 16, 2017

Conversation

bboreham
Copy link
Collaborator

Make low-level parsing more closely aligned to what we need, to save memory allocations.
See individual commit messages for more details.

These routines are called a lot, so this change reduces garbage collection

Benchmarks before (c5bdebd):

BenchmarkTopologyList-2         	       2	 603197223 ns/op	112161732 B/op	 1253503 allocs/op
BenchmarkTopologyHosts-2        	       3	 385415232 ns/op	82957173 B/op	 1015204 allocs/op
BenchmarkTopologyContainers-2   	      10	 192766565 ns/op	33145659 B/op	  407698 allocs/op

after this PR:

BenchmarkTopologyList-2         	       3	 484253544 ns/op	104023597 B/op	  977911 allocs/op
BenchmarkTopologyHosts-2        	       3	 366672131 ns/op	75139373 B/op	  742471 allocs/op
BenchmarkTopologyContainers-2   	      10	 178938068 ns/op	29289833 B/op	  272342 allocs/op

render/id.go Outdated
@@ -66,7 +65,8 @@ func externalNodeID(n report.Node, addr string, local report.Networks) (string,

// If the dstNodeAddr is not in a network local to this report, we emit an
// internet pseudoNode
if ip := net.ParseIP(addr); ip != nil && !local.Contains(ip) {
var into [5]byte // one extra byte to save a memory allocation in critbitgo
if ip := report.ParseIPb([]byte(addr), into[:4]); ip != nil && !local.Contains(ip) {

This comment was marked as abuse.

This comment was marked as abuse.


// ParseIP parses s as an IP address into a byte slice if supplied, returning the result.
// (mostly copied from net.ParseIP, modified to save memory allocations)
func ParseIPb(s []byte, into []byte) net.IP {

This comment was marked as abuse.

if second == -1 {
return "", "", "", false
}
return endpointNodeID[:first], endpointNodeID[first+1 : first+1+second], endpointNodeID[first+1+second+1:], true

This comment was marked as abuse.

This comment was marked as abuse.

These routines are called a lot, so this change reduces garbage collection
Pass in a slice on the stack instead of allocating one on the heap:
reduces garbage, hence makes the program run faster

Also apply knowledge that critbitgo will do an append() with one extra
byte, so we do that allocation up-front too.  This is innocuous should
we stop using critbitgo or should its internals change.
@bboreham bboreham force-pushed the parse-optimisations branch from 41e33ed to b989006 Compare November 15, 2017 23:17
@bboreham bboreham merged commit a375891 into master Nov 16, 2017
@dlespiau dlespiau deleted the parse-optimisations branch November 21, 2017 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants