fix(better_route_back): Use target peer if exists

[mfornet]

In routed message that uses hash to route message back, add also target peer to it, so message are routed optimally when the path to such peer is known.

This upgrade change layout of the message, so extra work is required to make it backward compatible.

Fixes #2989

Most of dropped messages happens because the combination of routed back heuristic and rebalancing strategy. This issue is solved in this PR.

Test plan

1. Existing tests (rust+nightly) extensively cover routing mechanism.
2. Is backward compatible

[gitpod-io]

[bowenwang1996]

It feels to me that we are making this unnecessarily complex. It should be rare when we do not know the author of the message. If that is the case, we can afford to drop the message.

[SkidanovAlex]

@mfornet do we actually cover cases in nightly in which the sender of a message is not a known peer?
We definitely need targeted tests for this change.

[mfornet]

@mfornet do we actually cover cases in nightly in which the sender of a message is not a known peer?
We definitely need targeted tests for this change.

Yes, it is both covered in simplified rust test (useful for debugging) and nightly test (those with blacklisted peers)

[mfornet]

It feels to me that we are making this unnecessarily complex. It should be rare when we do not know the author of the message. If that is the case, we can afford to drop the message.

@bowenwang1996 I don't think we should discard route back mechanism. It is useful for new peers connecting to the network, thinking mostly that we have a protection mechanism against an adversary that creates millions of ghost identities, and even in this case, the routing mechanism is still working.

[bowenwang1996]

Okay, but it seems to me that for the messages that require route back, we can afford to lose some of them. I looked at the messages that need a response, and the only relevant ones right now are PartialEncodedChunkResponse and StateResponse. For both we have retry mechanism, so as long as the message loss happens only occasionally, I don't think it is a huge concern.

[mfornet]

[discussion] This PR is not backward compatible atm, and making it backward compatibly will introduce a lot of complexity to the code, I propose to make this protocol update and after Phase 1.5 rethink how to make such big upgrades easily without messing the code as proposed in near/NEPs#95

[bowenwang1996]

We can postpone it for now unless we see that this is causing some real performance issues. However, if we just drop routeback completely it seems that it will be much easier to upgrade (the message format won't change). Can we upgrade in that way and later add routeback back if it is necessary.

[frol]

Should we close or merge this?

[stale]

This PR has been automatically marked as stale because it has not had recent activity in the 2 weeks.
It will be closed in 3 days if no further activity occurs.
Thank you for your contributions.

[bowenwang1996]

Assigning this to @pmnoxx since we have been seeing a lot of routeback not found error on mainnet

[akhi3030]

Given the age of the PR, I think it is no longer active.