Add needs-triage automation on k/k issues

[nikopen]

Following the discussion on https://groups.google.com/forum/#!topic/kubernetes-sig-contribex/BvGmOQ0v5f0 , a new label is proposed:

needs-sig-triage or similar.

Purpose of which is the following desired workflow:

                 |
            open bug/PR
                 |
                 V
        WAITING-ROOM: needs-sig, needs-sig-triage
                 |         ^
            (assign SIG)   |
                 |         |
                 V         |
      --> TRIAGE: needs-sig-triage<----
     |       /           \          |
     |  (close with   (verify)      |
     |   reason)          |         |
     |      |             V         |
      -- CLOSED      BACKLOG: kind/*, priority/*
                          |
                      (assign or claim)
                          |
                          V
                       IN-PROGRESS: assignee

i.e. - incoming new issue gets this label automatically, which signifies that it needs to be reviewed by the relevant SIG. After reviewing, it gets appropriate labels, is prioritized, or deleted if it's invalid / support request etc.

interrelated issue on how this will ultimately work - automatically triggered/de-triggered in kubernetes/community#3455

Open to suggestions on a different label and and overall comments.

[nikopen]

/assign @spiffxp @thockin @justaugustus @fejta @cjwagner @nikopen

[nikopen]

/sig release
/sig pm
/kind design

[thockin]

Thanks!. I will start using it immediately, even before any support for auto-assignment :)

[cjwagner]

I'm not sure this configuration does what you want based on the state machine diagram, but I'm also a bit confused by the terminology. My understanding is that this label is meant to be applied to PRs in the triage state in the diagram, but this will actually be applied to PRs in the waiting room as well. This could be avoided by changing the regexp to ^triage/|^needs-sig$.

This message also suggests removing the needs-review label, but doesn't say how. Labels shouldn't be manually added or removed by humans, especially this label since the automation will just replace the label if a human tries to remove it.

[nikopen]

Thanks, changed the label, regex and made the diagram clearer.

You're right about automatically removing the 'needs-sig-triage' label, and I wonder what the trigger would be. Perhaps by applying a 'lifecycle' label as in here: kubernetes/community#3455? That should be decided for the bigger picture

[thockin]

I don't mind if the label is applied in the waiting room, it's equally true. That said, having been sig-triaged without a sig is a weird state to be left in.

[nikopen]

we're thinking of utilizing lifecycle labels as status indicators (lifecycle/ready to be worked on - lifecycle/active in progress - forgottenfrozen - stale etc), with exit criteria of needs-triage to be
regexp: ^(lifecycle)/(ready|active|frozen)$

[nikhita]

/hold

This needs to be broadly communicated: https://github.com/kubernetes/community/blob/master/sig-contributor-experience/charter.md#cross-cutting-and-externally-facing-processes

Will take a look tomorrow (my timezone). also cc @kubernetes/sig-contributor-experience-pr-reviews

[thockin]

Status on this?

[cblecker]

/hold
Explicit hold, do not remove.

• I don't think we should proceed with this during 1.15 freeze.
• We still need to come to consensus on the larger triage workflow, and that hasn't happened. The right forum for that is contribex.
• I suggest we add it to the agenda for an upcoming meeting, and discuss in detail the overall plan. Once we have a plan, then we can flip the switch on the new workflow right at the beginning of the 1.16 cycle.
• From a technical perspective, new lifecycle labels need to be added to the plugin to be able to be applied:

  test-infra/prow/labels/labels.go

  Lines 33 to 36 in 46dacc9

  	LifecycleActive = "lifecycle/active"
  	LifecycleFrozen = "lifecycle/frozen"
  	LifecycleRotten = "lifecycle/rotten"
  	LifecycleStale = "lifecycle/stale"

How does that sound?

[nikopen]

@spiffxp I believe that since this feature solves a problem where currently there's no solution at all, and people are explicitly requesting for it (SIG leads in particular) it can't be called an experiment.

re: stale / fallback every 90 days, I have already suggested upping the interval to 120 or more so this happens less frequently.
However still,

1. if it indeed happens, then it means that the issue needs to be reviewed again and either put the frozen label or re-prioritize it. That's the sole purpose of the staleness feature of fejta-bot
2. the intent is that SIGs will quickly reap the benefits of the new incoming features in order to (ideally!!) achieve the following:

• better visibility over open issues
• easier optimization of current work capacity
• easier pre-planning over release cycles
• better awareness of issues' current status
• better outward visibility to external 'stakeholders' e.g. release team

and ultimately, less toil over issue triage and further management.

[nikopen]

@cblecker

• I think it can be enabled at any point in time given prior announcement and docs, it's not disruptive (needs-priority is often simply ignored) but perhaps people are very busy with freeze at this time.

• Consensus is desired, but avoiding excessive bikeshedding is also desired - there should be a middle ground and constructive comments to move this (and more) forward.

• re: meeting, it makes sense, though not sure if I'm able to attend this Wednesday evening, perhaps the next one.

• re: technical, looks like it, you know these details more than me! Do let us know of the extra details needed.

cheers!

[cblecker]

@nikopen Any time you're changing labels, automation, and workflows it's disruptive to contributors. As I've mentioned previously, I've been down this road before and received many complaints when we change things during important times in the cycle, or without excessive communication.

You're adding two extra labels to ~175 repos, and adding a comment requesting action being taken to 1550 open issues in k/k right now (https://github.com/kubernetes/kubernetes/issues?q=is%3Aopen+is%3Aissue+-label%3Alifecycle%2Factive+-label%3Alifecycle%2Ffrozen). This isn't an action to be taken lightly.

I think we already have some stuff on the agenda for this week's meeting (6/5) anyways, so maybe we want to pencil this in for discussion on our 6/12 meeting at 9:30am PT? Would that work for you?

[cblecker]

To be clear: I'm very happy that you're spearheading this, @nikopen. I very much would like a triage workflow that we can adopt project-wide. I'm just wanting to convey that these are large scope actions anytime we're talking about project-wide, or even k/k-wide changes. They impact a lot of issues and a lot of contributors.

[nikopen]

From that perspective indeed it seems like a bigger change! Happy that you're happy too.

Next week sounds good, let's speak soon

[cblecker]

Added to our agenda 👍

[spiffxp]

@nikopen I am also happy you're spearheading this effort. I frame in terms of experiment to suggest that we define what our expected outcome is, try the thing, and adjust if the actual outcome isn't what we expected. You list a lot of great bullet points, I'm just asking... how will we know if those are actually happening?

re: my comment on lifecycle labels, I was trying to point out that the use of those lifecycle label implies re-triaging is necessary every 90 days, and it wasn't clear to me that's what the intended outcome of the discussion was I've seen this far. Perhaps we want to contrast that with a label that isn't auto-removed, to imply that triaged-once is good enough.

[justaugustus]

/priority important-soon

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[nikopen]

/remove-lifecycle rotten

[justaugustus]

@nikopen -- Any plans to pick this back up for 1.18?
/milestone v1.18

[justaugustus]

Superseded by #16298, since this has stalled out.
/close

[k8s-ci-robot]

@justaugustus: Closed this PR.

In response to this:

Superseded by #16298, since this has stalled out.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@nikopen: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.