Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support local logging of events caused by external actors #679

Closed
7 tasks done
jfallows opened this issue Jan 2, 2024 · 1 comment · Fixed by #755
Closed
7 tasks done

Support local logging of events caused by external actors #679

jfallows opened this issue Jan 2, 2024 · 1 comment · Fixed by #755
Assignees
Labels
story Feature description from user's perspective

Comments

@jfallows
Copy link
Contributor

jfallows commented Jan 2, 2024

Describe the desired outcome from the user's perspective
As devops, I want to understand why zilla is not behaving as expected when integrating with external actors such as authenticating clients, Kafka broker authorization or API version support, or DNS servers, so that I can correct either zilla.yaml or the external system setup as needed.

Acceptance criteria

  • stdout exporter of HTTP request events (including authorized identity, if any)
  • stdout exporter of HTTP authorization failure events
  • stdout exporter of MQTT authentication failure events
  • stdout exporter of TLS authentication failure events
  • stdout exporter of DNS resolution failure events
  • stdout exporter of Kafka authentication failure events
  • stdout exporter of Kafka API version rejection events

Additional context
Events are recorded by the appropriate bindings and the telemetry stdout exporter logs locally.

Tasks

  • Define typed events for Kafka, HTTP, MQTT, TLS, TCP in .idl
  • Design append-only per-core binary log streams to record named events
  • Implement append-only per-core binary log streams to record events
  • Record configured named events for HTTP, MQTT, TLS, TCP (DNS), Kafka
  • Support generic configuration for telemetry exporter for named events
  • Implement the stdout telemetry exporter for named events
  • Update reference docs to include stdout telemetry exporter and named events

Notes

  • ensure logging on the data path cannot interfere with exporting logs locally (or remotely)
  • eventsN alongside dataN (per core)
    • structured types defined per binding / guard / vault / exporter / metric
    • simple binary messages
    • type indicators for deserialization
    • exporting can be lossy, need to know how to skip ahead when falling behind
    • rotating append-only binary logs
@jfallows jfallows added the story Feature description from user's perspective label Jan 2, 2024
@jfallows jfallows changed the title Support local stdout logging of events caused by external actors Support local logging of events caused by external actors Jan 2, 2024
@jfallows jfallows mentioned this issue Jan 2, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
story Feature description from user's perspective
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants