Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logging: better timestamp precision / log when CNI request finishes processing #381

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

maiqueb
Copy link
Collaborator

@maiqueb maiqueb commented Aug 17, 2023

What this PR does / why we need it:
The old format - RFC3339 - does not have enough precision - we need to see milliseconds (at least) to understand how long does it take for a CNI ADD / DEL to be processed in the IPAM stage to understand how long does it take for a CNI ADD / DEL to be processed in the IPAM stage.

A new log in added when the CNI request ( ADD or DELETE) finishes processing. It prints how long it took to process the request.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer (optional):
Old format:

2023-08-17T10:13:19Z [debug] Beginning IPAM for ContainerID: 5c4a2a24ef14b4c35fd55f553ffb7fcb6c05e493f2ba2f6d108b61cdf969cc86
2023-08-17T10:13:19Z [debug] Started leader election
2023-08-17T10:13:19Z [debug] OnStartedLeading() called
2023-08-17T10:13:19Z [debug] Elected as leader, do processing

New format:

2023-08-17T12:57:03.831205669Z [debug] OnStoppedLeading() called
2023-08-17T12:57:03.831217323Z [debug] Finished leader election
2023-08-17T12:57:03.831225863Z [debug] IPManagement: [{ }],

The old format - RFC3339 - does not have enough precision - we need to
see milliseconds (at least) to understand how long does it take for a
CNI ADD / DEL to be processed in the IPAM stage to understand how long
does it take for a CNI ADD / DEL to be processed in the IPAM stage.

Signed-off-by: Miguel Duarte Barroso <[email protected]>
@maiqueb maiqueb requested a review from dougbtv as a code owner August 17, 2023 13:35
We were missing a clear log indicating the IPAM CNI process is
finishing. Now, it should be possible to understand how long a request
takes from the logs.

Signed-off-by: Miguel Duarte Barroso <[email protected]>
@maiqueb
Copy link
Collaborator Author

maiqueb commented Aug 17, 2023

/cc @oshoval @dougbtv @nicklesimba

@maiqueb maiqueb changed the title logging: use the RFC3399Nano timestamp format logging: better timestamp precision / log when CNI request finishes processing Aug 17, 2023
Copy link
Member

@oshoval oshoval left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks
Can we add a space before the time ? (maybe need to remove T as well)
2023-08-17T12:57:03.831205669Z -> 2023-08-17 12:57:03.831205669Z

defer func() { safeCloseKubernetesBackendConnection(ipam) }()
defer func() {
safeCloseKubernetesBackendConnection(ipam)
logging.Debugf("DEL - IPAM plugin finished. Took: %q Config: %+v", time.Since(startTime).String(), *ipamConf)
Copy link
Member

@oshoval oshoval Aug 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe in case of a failure better to change to finished (failed)
so we can filter the failure lines easily when benchmarking ?
unless it is not important because when benchmarking we dont run failures cases
but it is more accurate this way, because it is not finished, but finished with failure

if it is - then also on ADD please

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there's no point bench marking failed results.

BUT it makes sense to log the result (or a simplified version of it).

I'll act on this.

Comment on lines +36 to +39
defer func() {
safeCloseKubernetesBackendConnection(ipam)
logging.Debugf("ADD - IPAM plugin finished. Took: %q Config: %+v", time.Since(startTime).String(), *ipamConf)
}()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider using a function that gets a string, startTime and is reused in both places
as beside ADD / DEL string it is the same

Copy link
Member

@dougbtv dougbtv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this PR as it stands. Oshoval's comments are nice improvements for sure. Thanks Miguel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants