Refactor metric records logic

[bogdandrutu]

Changes in the PR:

• Cleanup the complicated logic of keeping track of active Records, by implementing a mechanism that guarantees that an entry can be referenced iff (if and only if) it is in the map as well, so records removed from the map will never see new updates.
• Removes the need of two different lists (primary and reclaimed), also the logic of moving entries between the lists.
• Removes the need of the epoch logic. A simple modified atomic boolean is enough.

This PR also:

• Fixes race condition when a reclaimed entry is added back to the primary and then removes another entry from the map (same mapKey). This can happen when an entry is initially removed (some usages still happens to move it to reclaimed), a new entry for the same mapKey is added to the map, so when the initial entry is added back to the primary list will remove the newly added entry from the map. If it happens that the newly entry is bound then we will have "stale" records that will be used only for that handle.
• Simplifies the code, there may be other edge-cases and hidden race conditions with a more complicated code.
• Improves performance on the critical path, by not requiring cas when unbind (and removing extra atomic lods/stores - memory barriers), a cas is added on the collection side but that happens async.

Possible future improvements:

• Understand better when to report changes for a Record: currently if a Bound is used the values are always reported even if no changes.
• Use a per instrument map/list that improve contention on the map and list (even if they are "lock free" data structures, in a lot of places we do cas while succeed).
• Consider to iterate over the lock free records list without always removing and adding back elements (2 cas operations).
• For readability: Isolate the lock free list logic (by creating a proper list class, instead of having methods on the singlePtr as well as on the SDK + Record.

[jmacd]

@krnowak please review :)

[jmacd]

@paivagustavo please review :)

[jmacd]

Looks nice. Got any new benchmark results? Thanks :)

[bogdandrutu]

@jmacd please tell me how to run them

[jmacd]

Should be just go test -bench=. in the package directory, before and after.

[bogdandrutu]

Give me 1h to understand why the stress test blocked exactly where I pointed that it may block :))

[jmacd]

Cool. This is now the second technique I've seen that uses an atomic integer plus one bit to accomplish lock freedom. Discussed here, and as seen in the Prometheus-Go client. Interestingly (or confusingly), the other technique uses the high bit while this technique uses the low bit.

[bogdandrutu]

@jmacd done you can see the last commit if you are interested where the bug was :). Interestingly I couldn't reproduce on my local machine.

[jmacd]

Cool. The stress test reproduced a number of bugs for me, during development, but it was always on CircleCI and never on my local machine. I think it has to do with limited CPU count?

[bogdandrutu]

@jmacd I have some comments in the future work, please let me know if you want to create an issue for them, also left some todos related. Would like to know if you want to track these and how.

[krnowak]

Mostly typos, but the busy waiting during instrument binding makes me uneasy.

[krnowak]

Found last small typo, otherwise looks good. Thanks!