Structured logs: add identifiers & tx hashes in log output

[adizere]

Crate

ibc-relayer

Summary of problem

It is currently very difficult to correlate lines of output in the Hermes log. The only support for doing so is the thread identifiers, which became useless once a relayer action (e.g, submitting to a chain) crosses thread boundaries. This is a problem because it's difficult to understand from the log the sequence of steps that Hermes takes towards accomplishing a task such as relaying an IBC ICS20 packet.

Proposal

High level description: Break down Hermes tasks into concrete steps. A single task, such as handing an event, involves multiple steps (all the way from receiving the event from the WS to the last part of submitting the tx to a chain). Each step may span multiple lines of output in the log. All of the steps and the corresponding log lines that pertain to a single task should have an identifier.

In addition to identifiers per task, we would also like to make tx hashes as first-order citizens in the log. Towards this end, at least one of the steps that Hermes takes towards handling an event it should output

Acceptance Criteria

• break Hermes tasks into discrete steps, and tag all steps & log output for a task with the same identifier;
• identifier reqs: can be a simple random string (not necessarily universally unique)
• info level log output detailing the tx hashes that are being handled as part of a task
• can the logs indicate activity (or lack thereof) in more clearer ways??

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate milestone (priority) applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned