[awss3exporter]: add Sumo Logic marshaler for logs in Installed Collector format #23649
Conversation
aboguszewski-sumo
changed the title
| dateVal := lr.ObservedTimestamp() | ||
| body := attributeValueToString(lr.Body()) | ||
|
|
||
| logEntry(&buf, "{\"date\": \"%s\",\"sourceName\":\"%s\",\"sourceHost\":\"%s\",\"sourceCategory\":\"%s\",\"fields\":{},\"message\":\"%s\"}", |
There was a problem hiding this comment.
Are you sure this is going to produce valid JSON? What if I have source field with the value "foo"bar" ? Can you please add a test with that example?
I note you're also dropping all resource and log attributes, is that expected or did I miss something?
There was a problem hiding this comment.
I think it's expected to drop these, but I'll confirm that with others.
There was a problem hiding this comment.
After verifying the format, I've decided to do the following:
- replace custom field format with JSON and quote it
- place all resource attributes that are not
source*intofields - merge log record-level attributes into one JSON with the body
Sorry for the mess, but it should be clearer now.
|
@atoulme sorry for the mess, but I've decided to rework some things in this PR. It's ready for another review. |
atoulme
added
the
ready to merge
aboguszewski-sumo
force-pushed
the
awss3exporter-add-sumo-logs-marshaller
branch
from
5543a19
to
fb2d997
Compare
|
One small change before merging: the file format was changed to |
|
needs a rebase |
TylerHelmuth
removed
the
ready to merge
…rmat Signed-off-by: Katarzyna Kujawa <[email protected]>
Signed-off-by: Katarzyna Kujawa <[email protected]>
aboguszewski-sumo
force-pushed
the
awss3exporter-add-sumo-logs-marshaller
branch
from
9277a8b
to
d084d23
Compare
Description: This PR adds a new marshaller for
awss3exporter. It exports logs in the format of Sumo Logic Installed Collector. Metrics and traces are not supported - creating an exporter for them will result in an error.Currently, nested typed (eg. map inside a map) might not be supported correctly - I have to confirm the IC's behavior with them, but I wanted to create the PR so that it can be reviewed early.
Link to tracking Issue: #23212
Testing: Unit tests and manual e2e tests. Some automatic e2e tests will come later, but they will not be part of this repo, they will be a test for integrating the ingest with Sumo Logic's backend.
Documentation: Readme updated.