Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Alert silences management #2511

Merged
merged 4 commits into from
May 6, 2020
Merged

Alert silences management #2511

merged 4 commits into from
May 6, 2020

Conversation

alexandre-allard
Copy link
Contributor

@alexandre-allard alexandre-allard commented May 4, 2020 •
edited
Loading

Component: salt

Context: See #2193

Summary:
Add a Salt module to manage alert silences.

Acceptance criteria:
After having deployed a bootstrap node, being able to put a silence on an alert, list it and then remove it, from the Salt master:

salt-run salt.cmd metalk8s_monitoring.get_alerts
salt-run salt.cmd metalk8s_monitoring.add_silence KubeMemOvercommit
salt-run salt.cmd metalk8s_monitoring.get_silences active
salt-run salt.cmd metalk8s_monitoring.delete_silence ${silence_id}

Closes: #2464

@alexandre-allard alexandre-allard requested a review from a team as a code owner May 4, 2020 08:09
@bert-e
Copy link
Contributor

bert-e commented May 4, 2020

Hello alexandre-allard-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

@bert-e
Copy link
Contributor

bert-e commented May 4, 2020

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • one peer

Peer approvals must include at least 1 approval from the following list:

@alexandre-allard alexandre-allard force-pushed the bugfix/2464-alert-silences-management branch from 509202a to 01c22df Compare May 4, 2020 08:42
Ebaneck
Ebaneck suggested changes May 4, 2020
View reviewed changes
salt/_modules/metalk8s_monitoring.py Outdated Show resolved Hide resolved
salt/_modules/metalk8s_monitoring.py Outdated Show resolved Hide resolved
salt/_modules/metalk8s_monitoring.py Outdated Show resolved Hide resolved
salt/_modules/metalk8s_monitoring.py Show resolved Hide resolved
@bert-e
Copy link
Contributor

bert-e commented May 4, 2020

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • one peer

Peer approvals must include at least 1 approval from the following list:

The following reviewers are expecting changes from the author, or must review again:

gdemonet
gdemonet suggested changes May 4, 2020
View reviewed changes
Copy link
Contributor

@gdemonet gdemonet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ingress (even control-plane) objects are for exposing services to the outside. From Salt, we shouldn't need it.

salt/_modules/metalk8s_monitoring.py Show resolved Hide resolved
salt/metalk8s/addons/prometheus-operator/deployed/ingress.sls Outdated Show resolved Hide resolved
salt/_modules/metalk8s_monitoring.py Outdated Show resolved Hide resolved
@bert-e
Copy link
Contributor

bert-e commented May 4, 2020

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • one peer

Peer approvals must include at least 1 approval from the following list:

The following reviewers are expecting changes from the author, or must review again:

@alexandre-allard alexandre-allard force-pushed the bugfix/2464-alert-silences-management branch from 01c22df to cfe348e Compare May 5, 2020 07:36
@alexandre-allard
salt: add a method to fetch a service endpoints
82d5589 
retrieve the endpoint for a given service, returning
the first IP address and a list of ports.

this method will then be used to retrieve alertmanager
endpoint and communincate with its API to manage
alerts and silences.

Refs: #2464
@alexandre-allard
salt: use the common method to fetch service endpoints
9ad7c60 
this method has been moved to a common place, so that
other modules can use it.

Refs: #2464
@alexandre-allard alexandre-allard force-pushed the bugfix/2464-alert-silences-management branch from cfe348e to a978aa3 Compare May 5, 2020 08:26
@alexandre-allard
salt: add metalk8s utils module
c5c5894 
add an helper method to automatically retry
http requests.

Refs: #2464
@alexandre-allard
salt: add new Salt module for monitoring
e85b979 
for now, this module only manages silences (add,
delete, list) through call to Alertmanager API.
It's also possible to list alerts.

Refs: #2464
@alexandre-allard alexandre-allard force-pushed the bugfix/2464-alert-silences-management branch from a978aa3 to e85b979 Compare May 5, 2020 09:44
@bert-e
Copy link
Contributor

bert-e commented May 6, 2020

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • one peer

Peer approvals must include at least 1 approval from the following list:

The following reviewers are expecting changes from the author, or must review again:

@bert-e
Copy link
Contributor

bert-e commented May 6, 2020

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • one peer

Peer approvals must include at least 1 approval from the following list:

@alexandre-allard
Copy link
Contributor Author

/approve

@bert-e
Copy link
Contributor

bert-e commented May 6, 2020

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/2.6

The following branches will NOT be impacted:

  • development/1.0
  • development/1.1
  • development/1.2
  • development/1.3
  • development/2.0
  • development/2.1
  • development/2.2
  • development/2.3
  • development/2.4
  • development/2.5

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented May 6, 2020

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/2.6

The following branches have NOT changed:

  • development/1.0
  • development/1.1
  • development/1.2
  • development/1.3
  • development/2.0
  • development/2.1
  • development/2.2
  • development/2.3
  • development/2.4
  • development/2.5

Please check the status of the associated issue None.

Goodbye alexandre-allard-scality.

@bert-e bert-e merged commit 48f73a1 into development/2.6 May 6, 2020
@bert-e bert-e deleted the bugfix/2464-alert-silences-management branch May 6, 2020 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@slaperche-scality slaperche-scality slaperche-scality left review comments

@Ebaneck Ebaneck Ebaneck approved these changes

@gdemonet gdemonet gdemonet approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Alert silences management
5 participants
@alexandre-allard @bert-e @Ebaneck @slaperche-scality @gdemonet