Add support for file-based configuration of logging #274
Conversation
Also support log rotation for the configured values. Changes to be committed: modified: README.md modified: src/logger.py modified: src/sidecar.py
|
@jekkel can you please review and share your comments. |
Also support log rotation for the configured values. Changes to be committed: modified: README.md modified: src/logger.py modified: src/sidecar.py
…bvc/k8s-sidecar into log_to_file_with_rotation
Also support log rotation for the configured values. Changes to be committed: modified: README.md modified: src/logger.py modified: src/sidecar.py
Also support log rotation for the configured values. Changes to be committed: modified: README.md modified: src/logger.py modified: src/sidecar.py
…bvc/k8s-sidecar into log_to_file_with_rotation
|
Hi Reviewers, can you please review and provide your comments |
|
@jekkel Can you please have a look at my Pull request? |
|
Hi @venkatbvc is there a specific log format you have in mind for the transformation you want to enable with this change? Just wondering whether we should provide another pre-configured format instead.... 🤔 |
|
@jekkel in our company we have a custom log format, unfortunately i cannot share the format. but it is in json format. So we have used a sidecar approach where we write logs of kiwigrid sidecar to a logfile(using klog runner :https://pkg.go.dev/k8s.io/component-base/logs/kube-log-runner) and are transforming the current format to what we need. but this solution also needs to rotate logfiles. So was thinking if we support writing to a file with log rotation then it will help people to align logs to other formats. |
|
@jekkel We can set the log configuration from file. where user has flexibility to configure the log handlers. If this approach is fine with you , i can change and push it. |
From my POV I'd rather configure the logging system than adding a transformation pipeline. In fact, there'sthe same use case for custom log formats apply for "native" cloud integration, be it GCP, AWS or whatever. For example does GCP allow for rich structured logs via JSON but for getting the most out of it one need to conform to certain conventions. So I'm looking forward to a generic log format configuration feature 👍 |
|
@jekkel Sure i will modify my changes according the config way. |
Changes to be committed: modified: README.md modified: src/helpers.py new file: src/log_config.yaml modified: src/logger.py modified: src/resources.py modified: src/sidecar.py modified: test/resources/change_resources.yaml modified: test/resources/resources.yaml
…bvc/k8s-sidecar into log_to_file_with_rotation
|
@jekkel I have made the configuration in a generic way. Please review and provide your comments. |
|
@jekkel Did you get a chance to go through the changes? |
|
Hi @tomrk-esteam8 could you please support here? |
|
Did you a chance to look at the code? |
|
It seems checks does not work. Please adjust tests. |
|
Yes I am working on it. |
|
@tomrk-esteam8 Can you please trigger checks. I have modified code to fix the issue. |
|
@tomrk-esteam8 All the checks have passed. Can you please check and merge if all is fine? |
|
@tomrk-esteam8 Did you get a chance to look at the changes? |
|
thanks for your contribution, could you please add some tests as well? |
There was a problem hiding this comment.
Awesome work, thanks for the contribution! From my PoV only some last bits need polishing, then this is good to go!
@tomrk-esteam8 IMO it's a minor version increment.
…bvc/k8s-sidecar into log_to_file_with_rotation
|
@jekkel I have addressed the comments. Can you trigger the build to see if the tests goes through well |
…bvc/k8s-sidecar into log_to_file_with_rotation
|
PLease take a look at the workflow run: https://github.com/kiwigrid/k8s-sidecar/actions/runs/5520729921/jobs/10068885896?pr=274 I guess the file logging sidecar should not be waited for (unless we configure it to also log on the console?) |
…bvc/k8s-sidecar into log_to_file_with_rotation
|
@tomrk-esteam8 @jekkel @ChristianGeie can you please trigger the checks. |
|
@jekkel Yes i modified and pushed the code. can you please trigger the checks. |
|
just triggered all checks |
…bvc/k8s-sidecar into log_to_file_with_rotation
|
@ChristianGeie Thanks, i see an issue in the syntax of the issue. I fixed it. Mostly it should go through. can you please trigger checks again. |
|
@venkatbvc sure |
|
All, Added debug commands to identify the failure cause in the test. Can you please trigger check |
…bvc/k8s-sidecar into log_to_file_with_rotation
|
@jekkel @tomrk-esteam8 @ChristianGeie Can you trigger the checks. Issue is fixed. |
|
@jekkel All checks passed. Can you merge if the changes are OK. |
jekkel
changed the title
Add support for writing logs to a file, console or both.
Also support log rotation for the configured values. Changes to be committed:
modified: README.md
modified: src/logger.py
modified: src/sidecar.py
This helps end users to transform logs to the required ones by using fluentd or any other mechanism