Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Structured logs for relayer logic #1491

Merged
merged 37 commits into from
Jan 17, 2022
Merged
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
a29c9c7
More structure in logs, pass 1
adizere Oct 25, 2021
4df293f
Pass 2
adizere Oct 25, 2021
5ae8e42
Pass 3
adizere Oct 25, 2021
50ff524
Resolving todos, refactoring
adizere Oct 25, 2021
794a7bf
Better config.toml comment
adizere Oct 26, 2021
cc5e158
Merge branch 'master' into adi/structured_logs
adizere Nov 26, 2021
d75c832
Post-merge fixes
adizere Nov 26, 2021
9d17f44
Merge branch 'master' into adi/structured_logs
mzabaluev Dec 17, 2021
a2c3311
Post-merge fix
mzabaluev Dec 17, 2021
5906035
Sketch: printing tx hashes from SendPacket events.
adizere Dec 20, 2021
74913b7
log the tx hashes in ibc_channel event SendPacket
mzabaluev Dec 21, 2021
dcf662a
Improve code to print out the tx hash
mzabaluev Dec 22, 2021
f9ca4db
Actually enter the tracing span
mzabaluev Dec 22, 2021
5de3f2b
Apply suggestions from code review
adizere Dec 23, 2021
1d15085
Comment explaining TrackedMsgs
adizere Dec 23, 2021
60e66e6
Removed use of TrackedEvents Display impl
adizere Dec 23, 2021
4d94d54
Merge branch 'master' into adi/structured_logs
mzabaluev Dec 23, 2021
d189d2c
Erase Display impl for TrackedMsgs
mzabaluev Dec 23, 2021
3166f76
Allow passing IDs without copy in TrackedMsgs
mzabaluev Dec 23, 2021
c93249b
Different tracking ids for creation flows
mzabaluev Dec 23, 2021
1bba81b
Redo displaying for OperationalData
mzabaluev Dec 23, 2021
6df6b63
Deabbreviate an info level log message
mzabaluev Dec 23, 2021
85c60fa
Improve logging of operational data
mzabaluev Dec 23, 2021
b8e9c7c
Merge branch 'master' into adi/structured_logs
mzabaluev Jan 5, 2022
d84bccd
Remove verbose wording on TrackedMsgs IDs
mzabaluev Jan 5, 2022
c5f7d87
Merge branch 'master' into adi/structured_logs
mzabaluev Jan 10, 2022
0507161
Fix typos in descriptions of RunError variants
mzabaluev Jan 10, 2022
914373c
Use a tracing span for task log messages
mzabaluev Jan 10, 2022
d1ae093
Rework tracing spans for background tasks
mzabaluev Jan 11, 2022
108d081
Merge branch 'master' into adi/structured_logs
mzabaluev Jan 11, 2022
e6b41f9
Erase Display impl on RelayPath, use spans instead
mzabaluev Jan 11, 2022
73d135e
Shorten or remove span IDs for supervisor tasks
mzabaluev Jan 12, 2022
6cb9023
Erase [rest] prefixes from log messages
mzabaluev Jan 12, 2022
c69aa81
Merge branch 'master' into adi/structured_logs
mzabaluev Jan 17, 2022
7ae8880
Simplification & consolidation w/ Mikhail
adizere Jan 17, 2022
bfec7d6
Changelog entry for #1491
mzabaluev Jan 17, 2022
45a9761
Merge branch 'master' into adi/structured_logs
mzabaluev Jan 17, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 11 additions & 9 deletions relayer/src/util/task.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use core::time::Duration;
use crossbeam_channel::{bounded, Sender};
use std::sync::{Arc, RwLock};
use std::thread;
use tracing::{error, info, warn};
use tracing::{error, info, span, warn};

use crate::util::lock::LockExt;

Expand Down Expand Up @@ -90,14 +90,19 @@ pub fn spawn_background_task<E: Display>(
interval_pause: Option<Duration>,
mut step_runner: impl FnMut() -> Result<Next, TaskError<E>> + Send + Sync + 'static,
) -> TaskHandle {
info!("spawning new background task {}", task_name);
let span = span!(tracing::Level::ERROR, "task", name = %task_name);
let _entered = span.enter();

info!("spawning");
adizere marked this conversation as resolved.
Show resolved Hide resolved

let stopped = Arc::new(RwLock::new(false));
let write_stopped = stopped.clone();

let (shutdown_sender, receiver) = bounded(1);
let thread_span = span.clone();

let join_handle = thread::spawn(move || {
let _entered = thread_span.enter();
loop {
match receiver.try_recv() {
Ok(()) => {
Expand All @@ -106,17 +111,14 @@ pub fn spawn_background_task<E: Display>(
_ => match step_runner() {
Ok(Next::Continue) => {}
Ok(Next::Abort) => {
info!("task is aborting: {}", task_name);
info!("aborting");
break;
}
Err(TaskError::Ignore(e)) => {
warn!("task {} encountered ignorable error: {}", task_name, e);
warn!("encountered ignorable error: {}", e);
}
Err(TaskError::Fatal(e)) => {
error!(
"aborting task {} after encountering fatal error: {}",
task_name, e
);
error!("aborting after encountering fatal error: {}", e);
break;
}
},
Expand All @@ -127,7 +129,7 @@ pub fn spawn_background_task<E: Display>(
}

*write_stopped.acquire_write() = true;
info!("task {} has terminated", task_name);
info!("terminated");
});

TaskHandle {
Expand Down