Node from stream backrefs optimisation

[matt-o-how]

Use a Vec<NodePtr> stack instead of NodePtr / SExps in node_from_stream_backrefs and add a new traverse_path_with_vec() function to handle backrefs

[coveralls-official]

Pull Request Test Coverage Report for Build 13266025427

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 238 of 238 (100.0%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.3%) to 94.141%

---

Totals
Change from base Build 12933937517:	0.3%
Covered Lines:	6298
Relevant Lines:	6690

---

💛 - Coveralls

[arvidn]

it looks correct, as far as I can tell. I think we need tests for all interesting cases, to make sure it works. I'm also interested in seeing a benchmark. Does this make a difference? I would expect it to at least use less memory, which typically means faster on small machines (like Raspberry PI)

[arvidn]

I think these things are still needed:

• preserve the existing function, partly to control when we switch over to the new one, and also to be able to test that both behave the same
• ensure the new function produce the same result as the old one, e.g. with a fuzzer.
• ensure the new function behave the same with regards to limits to the number of pairs created by Allocator. It can be tested in a fuzzer by building with the counters build feature
• benchmark to demonstrate that this is an improvement (this should probably be done early, as we might want to scrap this idea if it doesn't carry its weight)
• survey the mainnet and testnet blockchains to see if back references into the parse-stack eveer exists in the wild
• unit tests for all edge cases

[arvidn]

These are the benchmark results I get on raspberry Pi 5:

deserialize/node_from_bytes_backrefs-compressed
                        time:   [352.36 µs 358.58 µs 367.28 µs]
deserialize/node_from_bytes_backrefs_old-compressed
                        time:   [344.03 µs 350.53 µs 358.28 µs]

So, a slight performance degradation. I would have expected it to be a slight improvement on a system with small caches. But presumably the memory savings is worth it. Do we have a measurement on that?