x/build: submit queue, possibly implemented as optional auto-submit on +2 when trybots/slowbots with rebase pass

[minux]

I see a trend of giving +2 together with trybot recently.
because we've defined that +2 means it's ok to submit,
I wonder if trybot could just submit the change (on behave
of the user who gives +2 and also invokes the trybot) if
the trybot runs are successful.

what do you think?

[bradfitz]

Russ does not want automatic submission for his own CLs, but he's fine with it for others.

We could add a new checkbox.

[bradfitz]

/cc @adg

[minux]

yeah, adding a new checkbox for auto-submit is fine too (actually I think we can make the 2nd checkbox do both: invoke the trybot and submit if it passes.)

[adg]

Until we can have the trybots automatically rebase on the HEAD before running, I don't think this is wise. Am I being paranoid?

[mwhudson]

On 4 September 2015 at 14:09, Andrew Gerrand [email protected]
wrote:

Until we can have the trybots automatically rebase on the HEAD before
running, I don't think this is wise. Am I being paranoid?

I don't think so.

[minux]

On Thu, Sep 3, 2015 at 10:09 PM, Andrew Gerrand [email protected]
wrote:

Until we can have the trybots automatically rebase on the HEAD before
running, I don't think this is wise. Am I being paranoid?

If that could cause problems, then it will also affect us now because there
is no
way for a human reviewer to notice anything unusual before clicking the
submit
button. The trybot could do a few more checks before submission though.

If we do want to implement, we need to make sure that the trybot will only
try to
submit the revision that the trybot has been run on.

[bradfitz]

See also: #9858 (rebase when testing)

[josharian]

Here's a slightly more fleshed out suggestion, for discussion.

Add an AutoSubmit score. When set to +1, it requests that GopherBot submit when the following conditions hold.

• The CL has a codereview +2.
• The AutoSubmit score was set to +1 simultaneously to or after the CL received a codereview +2. (This is to ensure that the choice to submit was made after seeing an approved form of the CL.)
• 24 hours have passed since the AutoSubmit score was set, to give others a chance to comment.
• There have been no negative codereview scores since the AutoSubmit score was set to +1 (for the final time).

Once all those have been met, gopherbot does a trybot run after rebasing. If it passes, it submits. If it fails, then a human must intervene.

[josharian]

cc @dmitshur @bcmills

[bcmills]

• The AutoSubmit score was set to +1 simultaneously to or after the CL received a codereview +2. (This is to ensure that the choice to submit was made after seeing an approved form of the CL.)

I don't think that's necessary: presumably the reviewer can see the AutoSubmit score ahead of time, so if they want to explicitly “+2 with requested changes” they can revoke the AutoSubmit (perhaps by voting AutoSubmit -1?) at that point.

If the uploader of the CL does not have submit permission, then uploading a new snapshot should also revoke the AutoSubmit +1.

[bcmills]

• There have been no negative codereview scores since the AutoSubmit score was set to +1 (for the final time).

This condition seems a bit too strong: a -1 or -2 on the current patchset should certainly cancel auto-submit, but if someone votes -1 or -2 and the CL is subsequently reworked (and those negative votes removed), then it should not be necessary to clear and re-set the AutoSubmit +1 in order for it to take effect.

[josharian]

if they want to explicitly “+2 with requested changes” they can revoke the AutoSubmit

That is true, but I'd rather the default be safe.

if someone votes -1 or -2 and the CL is subsequently reworked

Again, that is true, but I'd rather be conservative, at least to start.

These minor disagreements notwithstanding, I'd be happy with just about any variant of this.

[dmitshur]

/cc @FiloSottile who also wanted this.

[FiloSottile]

Yup, and I also expected it to be an AutoSubmit vote. I'd like to send simple CLs by git codereview mail -r [email protected] -trybot -autosubmit and not have to think about them again, so I'd also relax the "AutoSubmit must come after Code-Review" requirement.

[matloob]

cc @andybons @dmitshur

I would love a submit queue. Ideally we could run all the trybots (including slowbots) when we actually submit a CL. (And if we're more like a submit queue and do a rebase when submitting, the results can be copied straight to build.golang.org when the tests pass and the CL is submitted).

Personally I'd prefer to not have autosubmit but instead have the submit button trigger a test run, submitting if the tests pass, but i'd be pretty happy with autosubmit too.

[dmitshur]

CC @rolandshoemaker.