Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logging Handler #468

Merged
merged 8 commits into from
Sep 29, 2022
Merged

Logging Handler #468

merged 8 commits into from
Sep 29, 2022

Conversation

tsloughter
Copy link
Member

This is a very simple version of what is needed to support the logging spec and is in part duplicating code from the batch span processor.

It has no overload protection and this will wait until we know what is planned for log sampling in the spec as it will overlap.

Oh, and no tests. I've only manually tested by bringing up the collector and a rebar3 shell in which I add the handler and then see logs going to the collector through the collector's console output where it will log each log it receives.

Tristan Sloughter and others added 2 commits September 23, 2022 17:19
This is a very simple version of what is needed to support the
logging spec and is in part duplicating code from the batch
span processor.

It has no overload protection and this will wait until we know
what is planned for log sampling in the spec as it will overlap.
@codecov
Copy link

codecov bot commented Sep 23, 2022

Codecov Report

Base: 73.61% // Head: 36.34% // Decreases project coverage by -37.27% ⚠️

Coverage data is based on head (8155c59) compared to base (446313a).
Patch coverage: 3.70% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #468       +/-   ##
===========================================
- Coverage   73.61%   36.34%   -37.28%     
===========================================
  Files          56       59        +3     
  Lines        1751     3555     +1804     
===========================================
+ Hits         1289     1292        +3     
- Misses        462     2263     +1801     
Flag Coverage Δ
api 68.43% <0.00%> (-0.34%) ⬇️
elixir 18.67% <0.00%> (-0.10%) ⬇️
erlang 36.29% <3.70%> (-38.82%) ⬇️
exporter 8.03% <3.88%> (-65.25%) ⬇️
sdk 78.91% <0.00%> (-0.10%) ⬇️
zipkin 51.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
apps/opentelemetry/src/otel_batch_processor.erl 76.36% <0.00%> (ø)
apps/opentelemetry/src/otel_exporter.erl 50.00% <0.00%> (-1.36%) ⬇️
apps/opentelemetry_api/src/opentelemetry.erl 77.41% <0.00%> (-2.59%) ⬇️
...ntelemetry_exporter/src/opentelemetry_exporter.erl 61.74% <0.00%> (-4.45%) ⬇️
...ter/src/opentelemetry_exporter_logs_service_pb.erl 0.00% <ø> (ø)
...emetry_exporter/src/opentelemetry_logs_service.erl 0.00% <0.00%> (ø)
apps/opentelemetry_exporter/src/otel_otlp_logs.erl 0.00% <0.00%> (ø)
...ps/opentelemetry_exporter/src/otel_otlp_common.erl 77.14% <50.00%> (-8.58%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

resource=Resource,
config=Config,
batch=Batch}) when map_size(Batch) =/= 0 ->
_ = export(Exporter, Resource, Batch, Config),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm noticing that this call here does not let the exporter update its state. If it could, we'd also be able to drastically simplify the FSM by letting export/4 do the optional call to init_exporter/1 and return the new config as well as letting the exporter update its state on return.

I however don't know what the future plans are for this module, so this drastic simplification possibly clashes with said plans.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting idea... I planned to simplify it by breaking out the shared logic from the batch_processor module. If that works there too then it makes sense to do.

?LOG_INFO("OTLP grpc export failed with HTTP status code ~s", [Status]),
error;
{error, Reason} ->
?LOG_INFO("OTLP grpc export failed with error: ~p", [Reason]),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any chance these logs messages are gonna be fed back into the OTel exporter and cause issues with write amplification?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh they definitely will

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a domain can be used here but then by default all these logs would be lost.. ugh

apps/opentelemetry_exporter/src/otel_otlp_common.erl Outdated Show resolved Hide resolved
apps/opentelemetry_exporter/src/otel_otlp_common.erl Outdated Show resolved Hide resolved
Copy link
Member

@ferd ferd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major concerns addressed. Still curious about the logging handler failing and possibly generating more logs to handle and the risk this might represent.

@tsloughter tsloughter force-pushed the log-sdk branch 3 times, most recently from e21f980 to 3c1205e Compare September 28, 2022 22:20
@tsloughter tsloughter merged commit 31296e9 into open-telemetry:main Sep 29, 2022
@tsloughter tsloughter deleted the log-sdk branch September 30, 2022 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants