Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add support for PG Query Log exporter #254

Merged
merged 8 commits into from
Oct 22, 2024

Conversation

bhupendray-yb
Copy link
Contributor

@bhupendray-yb bhupendray-yb commented Oct 15, 2024

Description:

Added support for DB Query logs exporter(https://docs.yugabyte.com/preview/yugabyte-cloud/cloud-monitor/logging-export/)

Changelog:

  • Added support for 4 subcommands for db query logging:
    • Enable logging
    • Disable logging
    • Describe config
    • Update logging config
  • Added unit test for all commands

CLI logs:

Enabling query log exporter

~/Developer/ybm-cli on feat/db_query_log_exporter                                                                                                       took 4s at 12:21:08
> make build && ./ybm cluster db-query-logging enable \
--cluster-name "bhupendray-cluster" \
--integration-name bhupendray-datadog-logging \
--log-line-prefix "%m :%r :%u @ %d :[%p] : %a :" \
--log-min-duration-statement 30 \
--log-connections true \
--log-duration false \
--log-error-verbosity DEFAULT \
--log-statement MOD
go build -ldflags="-X 'main.version=v0.1.0'" -o ybm
A newer version is available. Please upgrade to the latest version v0.1.20
State      Integration ID                            Log Config
ENABLING   7eb1b884-600e-4a91-a92c-9ee79b754e11   {"debug_print_plan":false,"log_connections":true,"log_disconnections":false,"log_duration":false,"log_error_verbosity":"DEFAULT","log_line_prefix":"%m :%r :%u @ %d :[%p] : %a :","log_min_duration_statement":30,"log_min_error_statement":"ERROR","log_statement":"MOD"}

Describe log exporter config:

~/Developer/ybm-cli on feat/db_query_log_exporter                                                                                                       took 7s at 12:09:00
> make build && ./ybm cluster db-query-logging describe --cluster-name "bhupendray-cluster"
go build -ldflags="-X 'main.version=v0.1.0'" -o ybm
A newer version is available. Please upgrade to the latest version v0.1.20
State       Integration ID                            Log Config
DISABLING   7eb1b884-600e-4a91-a92c-9ee79b754e11   {"debug_print_plan":false,"log_connections":true,"log_disconnections":false,"log_duration":false,"log_error_verbosity":"DEFAULT","log_line_prefix":"%m :%r :%u @ %d :[%p] : %a :","log_min_duration_statement":30,"log_min_error_statement":"ERROR","log_statement":"MOD"}

Disable query logs:

~/Developer/ybm-cli on feat/db_query_log_exporter                                                                                                           INT at 12:08:37
> make build && ./ybm cluster db-query-logging disable \
--cluster-name "bhupendray-cluster"
go build -ldflags="-X 'main.version=v0.1.0'" -o ybm
A newer version is available. Please upgrade to the latest version v0.1.20
Disabling DB query log config for the cluster, this may take a few minutes...
You can check the status via $ ybm cluster db-query-log-exporter describe --cluster-name bhupendray-cluster

Edit log exporter config:

~/Developer/ybm-cli on feat/db_query_log_exporter                                                                                                       took 5s at 11:19:37
> make build && ./ybm cluster db-query-logging update-config \
--cluster-name "bhupendray-cluster" \
--integration-name bhupendray-datadog-logging \
--log-line-prefix "%m :%r :%u @ %d :[%p] :" \
--log-min-duration-statement 60 \
--log-connections false \
--log-duration true \
--log-error-verbosity DEFAULT \
--log-statement ALL
go build -ldflags="-X 'main.version=v0.1.0'" -o ybm
A newer version is available. Please upgrade to the latest version v0.1.20
State     Integration ID                            Log Config
ACTIVE    7eb1b884-600e-4a91-a92c-9ee79b754e11  {"debug_print_plan":false,"log_connections":false,"log_disconnections":false,"log_duration":true,"log_error_verbosity":"DEFAULT","log_line_prefix":"%m :%r :%u @ %d :[%p] :","log_min_duration_statement":60,"log_min_error_statement":"ERROR","log_statement":"ALL"}

@bhupendray-yb bhupendray-yb requested a review from a team as a code owner October 15, 2024 06:36
Copy link

@bansal01yash bansal01yash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit.
Apart from that please squash commits before merging once you get approval from owners.

cmd/cluster/log-exporter/query_log_exporter.go Outdated Show resolved Hide resolved
Copy link
Contributor

@arishta-yb arishta-yb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left minor comments, mostly around readability and adding safety checks. Also, squash all the commits before merging to keep the commit history clean :)

@bhupendray-yb
Copy link
Contributor Author

I will resolve the comments.

Squash the commit

@bansal01yash @arishta-yb do I need to do it manually? In merge PR option I see, it's automatically going to squash and then merge.

@bhupendray-yb bhupendray-yb merged commit f2b9648 into main Oct 22, 2024
2 checks passed
@bhupendray-yb bhupendray-yb deleted the feat/db_query_log_exporter branch October 22, 2024 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants