Buffer connectable log devices #35
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* master: Version bump [ci skip] Add note about log level getting changed to troubleshooting section [ci skip] Support for logging to Kafka Version bump [ci skip] Fixed logger spec Added troubleshooting notes [ci skip] Updated changelog and contributor list Logger spec cleanup Cleanup of the configuration feature Check for config to be #call-able, not just non-nil Customize events with a blocK No need to explicitly check for String type Allow custom_fields to be strings or callable Update README.md Logging custom metadata, with tests and documentation Add appname to logged json object
|
@dwbutler Will this one get released? |
|
@hwo411 I do plan to release this eventually, when I find the time to revive this branch. If I bring it up to date with master, would you be willing to try out the branch and let me know how well it works for you? |
* master: (40 commits) Version bump [ci skip] Remove unnecessary tagged logging implementation from MultiLogger Version bump [ci skip] Support for logging to syslog Version bump + docs [ci skip] Readme cleanup [ci skip] Cleanup Fixes support for tagged logging in Multi Logger Fixes support for tagged logging in custom formatters Fixes MultiLogger under Rails 3.2 Support for multiple loggers Documentation update [ci skip] Update to prettier badges [ci skip] Codecov badge [ci skip] Set up Codecov Add note about known issue with UDP output [ci skip] Updated contributor list [ci skip] Version bump [ci skip] Use Travis CI container architecture Updated readme [ci skip] ... Conflicts: lib/logstash-logger/device/redis.rb
dwbutler
added a commit
that referenced
this pull request
Sep 21, 2015
Buffer connectable log devices
|
Released in 0.15.0. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This throws connectable log devices (such as TCP, UDP, and Redis) behind
Stud::Bufferin an attempt to improve reliability. In my testing, I have verified that log messages do get buffered, and will retry if an exception is thrown.However, I have found that sockets don't always raise an exception right away when the server side of the socket is closed. See http://stackoverflow.com/questions/2530652/ruby-tcpsocket-doesnt-notice-it-when-server-is-killed and https://www.ruby-forum.com/topic/148532. This means that some messages will inevitably get lost. Logstash does not return any sort of reply to acknowledge that it received the message, so it's impossible to tell if the message was sent successfully if no exception is raised.
Log messages are written out one at a time. This way, if an exception is raised, the whole batch of messages is retried, and messages are less likely to be lost. However, this does result in duplicate log messages being delivered. Logstash's uuid filter looks like a promising way of dealing with duplicate log messages.
This should fix #34