Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use logger interceptor in Temporal workers #1015

Merged
merged 1 commit into from
Sep 4, 2024

Conversation

jraddaoui
Copy link
Collaborator

Follow https://enduro.readthedocs.io/dev-manual/logging recommendations.

  • Use go.artefactual.dev/tools/temporal to set the clients logger.
  • Use replay-aware Temporal SDK logger from processing workflow.
  • Add logger interceptor to all workers.
  • Use go.artefactual.dev/tools/temporal GetLogger in activities.
  • Remove logger usage from local activities.

Refs #1000.

@jraddaoui jraddaoui self-assigned this Sep 2, 2024
Copy link

codecov bot commented Sep 2, 2024

Codecov Report

Attention: Patch coverage is 54.68750% with 29 lines in your changes missing coverage. Please review.

Project coverage is 53.11%. Comparing base (8aa0da8) to head (0ecec2a).
Report is 64 commits behind head on main.

Files with missing lines Patch % Lines
cmd/enduro-am-worker/main.go 0.00% 10 Missing ⚠️
cmd/enduro-a3m-worker/main.go 0.00% 7 Missing ⚠️
cmd/enduro/main.go 0.00% 7 Missing ⚠️
internal/workflow/processing.go 50.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1015      +/-   ##
==========================================
- Coverage   53.13%   53.11%   -0.02%     
==========================================
  Files         101      100       -1     
  Lines        5814     5810       -4     
==========================================
- Hits         3089     3086       -3     
+ Misses       2474     2471       -3     
- Partials      251      253       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@sevein sevein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I had not realized we were not doing this!

@@ -22,6 +22,7 @@ import (
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/spf13/pflag"
"go.artefactual.dev/tools/log"
temporal_tools "go.artefactual.dev/tools/temporal"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you like, you can enforce this alias here:

enduro/.golangci.yml

Lines 41 to 47 in 8aa0da8

alias:
- pkg: go.temporal.io/sdk/contrib/(\w+)
alias: temporalsdk_contrib_$1
- pkg: go.temporal.io/sdk/(\w+)
alias: temporalsdk_$1
- pkg: go.temporal.io/api/(\w+)
alias: temporalapi_$1

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like to enforce it.

@@ -212,6 +213,9 @@ func main() {
EnableSessionWorker: true,
MaxConcurrentSessionExecutionSize: cfg.A3m.Capacity,
MaxConcurrentActivityExecutionSize: 1,
Interceptors: []temporalsdk_interceptor.WorkerInterceptor{
temporal_tools.NewLoggerInterceptor(logger.WithName("a3m-worker")),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logger was already named enduro-a3m-worker (using appName) so I believe that the name of this child logger would be enduro-a3m.worker.a3m-worker, is that what you want?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, maybe not appending anything works better in this case.

@@ -57,7 +51,8 @@ func (a *UploadTransferActivity) Execute(
ctx context.Context,
params *UploadTransferActivityParams,
) (*UploadTransferActivityResult, error) {
a.logger.V(1).Info("Execute UploadTransferActivity", "SourcePath", params.SourcePath)
logger := temporal_tools.GetLogger(ctx)
logger.V(1).Info("Execute UploadTransferActivity", "SourcePath", params.SourcePath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A thought for the future: I've always seen this logging entry as unnecessary becuse the SDK produces a similar event automatically if I remember correctly, it does not log local variables or parameters but inputs are available in the Temporal workflow history anyways.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sevein I find the log entry helpful for debugging failures where Temporal schedules the activity (so it shows up in the workflow history) but the worker doesn't actually run the Execute() method. Echoing the parameters is probably unnecessary in that case though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps that can be done from the worker interceptor too (ActivityInboundInterceptor), I will investigate.

Follow https://enduro.readthedocs.io/dev-manual/logging recommendations.

- Use `go.artefactual.dev/tools/temporal` to set the clients logger.
- Use replay-aware Temporal SDK logger from processing workflow.
- Add logger interceptor to all workers.
- Use `go.artefactual.dev/tools/temporal` `GetLogger` in activities.
- Remove logger usage from local activities.
@jraddaoui jraddaoui force-pushed the dev/issue-1000-logger-interceptor branch from 8ca0b45 to 0ecec2a Compare September 4, 2024 00:55
@jraddaoui jraddaoui merged commit 0ecec2a into main Sep 4, 2024
13 checks passed
@jraddaoui jraddaoui deleted the dev/issue-1000-logger-interceptor branch September 4, 2024 00:56
@jraddaoui
Copy link
Collaborator Author

Thanks @sevein!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants