Customizable logging for message parts?

[reidsunderland]

The current logging that shows accepted and posted messages comes from flowcb/log https://github.com/MetPX/sarracenia/blob/development/sarracenia/flowcb/log.py.

In some cases, it would be useful to show additional information like the sundew_extension, size, checksum, pubTime, but these aren't necessarily useful in all cases.

We have a logEvents option that makes it possible to fine tune which log messages show up. Maybe it's too much fine tuning, but we could add another option that that lets us define which part of the message we want to see in the log (add sundew_extension, etc. when useful).

We do have messageDebugDump, but it shows too much information for most cases, and the dictionary format is harder to read. I also just noticed that it's not documented in SR3 Options.

[petersilva]

copying stuff over from #1156...
1 option is to change the default logging to include:

• if present: sundew_extension
• if present: first seven (based on git example) chars of identity value, like so "id: ab13f12"
• if present: size (not humanized to avoid loss of precision.)

to ... post? or accept, or both? (the routines in sarracenia/flowcb/log are _messageAcceptStr and _messagePostStr

[petersilva]

I don't know if the changes to default would be enough to obviate the need for programmable fields.

the best part is no part ... Elon Musk

[petersilva]

so... looking at log.py, I see that _messagePostStr is not even called by default, it calls just messageIDStr which is literally just the URL... maybe we need an overarching logging verbosity setting? something like:

logVerbose 0    ... warnings errors...
logVerbose 1    ... info... with msgID str...
logVerbose 2    ... info ... with medium message info (selected fields.)
logVerbose 3    ... info ... with full message dumps.
logVerbose 4  ... debug... with msgIDstr...
logVerbose 5  .. debug... with medium message info
                   7  ... debug with full dumps.
logVerbose 8 ... turn on message protocol debugging... 

[petersilva]

I thought this would be a five minute job... but the flow tests use the log outputs, so it's means modifying them all...

[petersilva]

to avoid messing with the tests... just make sure the new fields are near the end.
sample post mod post log:

2024-08-28 13:00:40,127 [INFO] sarracenia.flowcb.log after_post posted to exchange: xflow_public topic: v03.post.20200105.WXO-DD.meteocode.atl.csv a file with baseUrl: h
ttp://localhost:8001/ relPath: 20200105/WXO-DD/meteocode/atl/csv/2020-01-05T03-00-01Z_FPHX14_r9zf_PA.csv size: 804 id: ruTzj2p

This is from static, so no sundew_extension present... but I really like the short id string.