[DISCUSS] Stop elasticmachine from reporting build results in PR's

[rudolf]

Now that Kibana CI results are reported via Github Checks, can we disable elasticmachine's comments on PR's to keep the PR discussion focussed?

[joshdover]

I'd be open for this. The only benefit I see to the comments is that it sends you an email/notification when the build finishes, but I personally don't use that in my workflow.

[rudolf]

The only benefit I see to the comments is that it sends you an email/notification when the build finishes

Please leave a 👍 if your daily workflow depends on getting email notifications about the build status.

[stacey-gammon]

I'm so torn on this issue, I definitely rely on the email notifications but I hate the mess that all those comments leave! I also like that it's an easy shortcut straight to the ci. I often click the link from the email on my phone to send me straight to ci rather than github.

Is there a way we could have the elasticmachine make only one comment and just edit it when it runs? So instead of a bunch, something like:

  [timestamp] ci failed - [link]
  [timestamp] ci failed - [link] 
  [timestamp] ci passed - [link]

though... I think updating an existing comment doesn't send an email...

The other benefit of that history is sometimes with a flaky test investigation it's nice to have all the past links, or even just a bug investigation... it's like oh wait why did that previous commit fail? And I can go see it with the link and compare against the current ci run results.

[Bamieh]

Personally I use them. I don't mind seeing them in a PR

[mattkime]

I'd like to see this happen but we've seen some stability problems with GitHub checks. Its not uncommon for a Check API request to fail and consequently report a failed CI run. We're trying to improve our retry logic - which shouldn't be hard but I haven't done the work to verify that its doing the correct thing. I guess I think it might be just a little early for this.

[mattkime]

Also - congrats to anyone who who can make github email spam readable. ;-)

[spalger]

Maybe a better solution would be to disable the GitHub comments left by the rudimentary Jenkins plugin, and instead send comments from a bot that deletes and summarizes old comments. Maybe something like internal infrastructure team issue 6274.

Edit: Deletes of a users own comments don't leave history entries, so this would mean that there is a single comment and new comments would trigger emails.

We could take control of these comments if we wanted to, and maybe share them with other teams if it worked out well for us.

[rudolf]

The other benefit of that history is sometimes with a flaky test investigation it's nice to have all the past links, or even just a bug investigation... it's like oh wait why did that previous commit fail? And I can go see it with the link and compare against the current ci run results.

The check marks next to each commit in the PR gives a quick visual history of build failures:

It seems like there's a number of people who want to get build notifications. Using Github comments -> Email is a simple system, but for me getting a notification via slack would be more useful. Don't know if it's technically possible to map from github usernames to slack usernames.

[Bargs]

Slack wouldn't be a suitable replacement for email for me. Slack notifications are too transient, I can look at one and then get distracted and forget I ever received it. Email works for me because I leave it in my inbox until I've triaged it, at which point I archive the email.

[lukeelmers]

I think @spalger's suggestion is the best of both worlds: Everyone still gets an email notification because elasticmachine will post a new comment on each build, but the conversation history isn't cluttered with a bunch of outdated build status messages.