Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make syslog logging useable #411

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

maxmoehl
Copy link
Member

@maxmoehl maxmoehl commented Apr 19, 2024

Summary

This PR corrects and improves the direct-to-syslog logging of access logs for Gorouter.

  • does not log an empty new line when writing to syslog
  • allows the operator to configure truncation on syslog, as syslog servers usually have a message limit
  • significant performance improvement to the syslog implementation

The syslog implementation in Go has not been maintained by the Go authors since around 2015. While there are forks of the syslog code, all of them are also dormant. This PR inlines and significantly rewrites the syslog implementation, focusing on the following performance improvements:

  • No use of the expensive fmt.Sprintf
  • Determine all "static" fields (e.g. the PID) on initialization, not per message
  • Correct handling of UTF-8 BOM according to RFC5424

Limitations:

  • Only TCP and UDP are supported, as described in the routing-release property for the syslog network. All other means (unix sockets, etc.) are not supported.

Backward Compatibility

Breaking Change? Yes

  • This PR moves the syslog implementation from the BSD-style syslog format, which is not standardised, to the RFC5424 format.

* do not log an empty new line when writing to syslog
* allow operator to configure truncation on syslog
@maxmoehl maxmoehl marked this pull request as ready for review November 20, 2024 14:53
@maxmoehl maxmoehl requested a review from a team as a code owner November 20, 2024 14:53
@maxmoehl
Copy link
Member Author

I forgot to update the breaking change indicator after pushing my latest change. This is technically breaking because we move from the BSD-style syslog format which is not standardised to the RFC5424 format. I'm not sure how man people actually use the syslog implementation today, over UDP each log line also inserts an empty log line due to the (previously dedicated) call to Write. Over TCP it did probably work, however, it caused gorouter to perform worse as the old implementation did unnecessary syscalls for each log line written and the formatting was done using fmt.

The go standard library implementation of log/syslog has been abandoned
and is no longer maintained. In our tests with syslog logging in
gorouter we noticed a significant performance penalty over file based
logging. However, with file based logging we frequently observe broken
logs because logrotate interferes with log reading / writing.

This commit inlines the relevant file from log/syslog and refactors the
implementation to be on-par with the file based logging.

Co-authored-by: Alexander Lais <[email protected]>
@ameowlia
Copy link
Member

@maxmoehl
Copy link
Member Author

The old format looks something like this:

<34>Oct 11 22:14:15 mymachine MESSAGE

and the new format is something like this:

<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - \ufeffMESSAGE

@maxmoehl
Copy link
Member Author

maxmoehl commented Nov 25, 2024

For testing use this config:

logging.syslog_addr: 127.0.0.1:514
logging.syslog_network: udp
enable_access_log_streaming: true
write_access_logs_locally: false

@maxmoehl
Copy link
Member Author

I've also opened cloudfoundry/routing-release#446

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

3 participants