Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LinuxTools builders fails to (mabye?) push at the end of PR builds #62811

Closed
alexcrichton opened this issue Jul 19, 2019 · 2 comments · Fixed by #62970
Closed

LinuxTools builders fails to (mabye?) push at the end of PR builds #62811

alexcrichton opened this issue Jul 19, 2019 · 2 comments · Fixed by #62970
Assignees
Labels
T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.

Comments

@alexcrichton
Copy link
Member

Discovered through #62766 the end of a build failed with:

2019-07-18T17:43:29.7626178Z Verifying status of book...
2019-07-18T17:43:29.7645772Z Verifying status of nomicon...
2019-07-18T17:43:29.7656763Z Verifying status of reference...
2019-07-18T17:43:29.7669719Z Verifying status of rust-by-example...
2019-07-18T17:43:29.7681535Z Verifying status of edition-guide...
2019-07-18T17:43:29.7694011Z Verifying status of rls...
2019-07-18T17:43:29.7707209Z Verifying status of rustfmt...
2019-07-18T17:43:29.7718929Z Verifying status of clippy-driver...
2019-07-18T17:43:29.7730859Z Verifying status of miri...
2019-07-18T17:43:29.7744226Z Verifying status of embedded-book...
2019-07-18T17:43:29.7755764Z Verifying status of rustc-guide...
2019-07-18T17:43:29.7857440Z Cloning into 'rust-toolstate'...
2019-07-18T17:43:30.5204588Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-18T17:43:30.5441945Z [master d757389] (linux CI update)
2019-07-18T17:43:30.5442055Z  1 file changed, 1 insertion(+)
2019-07-18T17:43:31.2635801Z remote: Invalid username or password.
2019-07-18T17:43:31.2636960Z fatal: Authentication failed for 'https://github.com/rust-lang-nursery/rust-toolstate.git/'
2019-07-18T17:43:31.5986192Z From https://github.com/rust-lang-nursery/rust-toolstate
2019-07-18T17:43:31.5986526Z  * branch            master     -> FETCH_HEAD
2019-07-18T17:43:31.6172922Z HEAD is now at 601d596 Merge pull request #13 from atouchet/links
2019-07-18T17:43:31.6293057Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-18T17:43:31.6464398Z [master 8d4ee6f] (linux CI update)
2019-07-18T17:43:31.6464464Z  1 file changed, 1 insertion(+)
2019-07-18T17:43:31.9791628Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-18T17:43:32.3359605Z From https://github.com/rust-lang-nursery/rust-toolstate
2019-07-18T17:43:32.3360019Z  * branch            master     -> FETCH_HEAD
2019-07-18T17:43:32.3502731Z HEAD is now at 601d596 Merge pull request #13 from atouchet/links
2019-07-18T17:43:32.3618961Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-18T17:43:32.3803151Z [master 55dd91f] (linux CI update)
2019-07-18T17:43:32.3803223Z  1 file changed, 1 insertion(+)
2019-07-18T17:43:32.6895480Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-18T17:43:34.0220883Z From https://github.com/rust-lang-nursery/rust-toolstate
2019-07-18T17:43:34.0221978Z  * branch            master     -> FETCH_HEAD
2019-07-18T17:43:34.0372608Z HEAD is now at 601d596 Merge pull request #13 from atouchet/links
2019-07-18T17:43:34.0478844Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-18T17:43:34.0653889Z [master 69cc905] (linux CI update)
2019-07-18T17:43:34.0654492Z  1 file changed, 1 insertion(+)
2019-07-18T17:43:34.3745720Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-18T17:43:34.7285854Z From https://github.com/rust-lang-nursery/rust-toolstate
2019-07-18T17:43:34.7286707Z  * branch            master     -> FETCH_HEAD
2019-07-18T17:43:34.7437374Z HEAD is now at 601d596 Merge pull request #13 from atouchet/links
2019-07-18T17:43:34.7553136Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-18T17:43:34.7732285Z [master 69cc905] (linux CI update)
2019-07-18T17:43:34.7732378Z  1 file changed, 1 insertion(+)
2019-07-18T17:43:35.0867854Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-18T17:43:37.4349692Z From https://github.com/rust-lang-nursery/rust-toolstate
2019-07-18T17:43:37.4350364Z  * branch            master     -> FETCH_HEAD
2019-07-18T17:43:37.4489321Z HEAD is now at 601d596 Merge pull request #13 from atouchet/links
2019-07-18T17:43:38.0902365Z ##[error]Bash exited with code '1'.

That could be a weird unauthenticated pull from GitHub but I suspect that it's a push which will inevitably fail. I think we haven't configured the toolstate builder to not do anything on PRs on push?

@alexcrichton alexcrichton added the T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. label Jul 19, 2019
@Xanewok
Copy link
Member

Xanewok commented Jul 20, 2019

I believe I hit this in when updating RLS in #62805 (comment)

@RalfJung RalfJung mentioned this issue Jul 21, 2019
@jonas-schievink
Copy link
Contributor

jonas-schievink commented Jul 23, 2019

So is it expected that all PR builds currently fail with this error on the LinuxTools builder?

EDIT: Ah nevermind, only PRs that update submodules.

@ehuss ehuss mentioned this issue Jul 24, 2019
@RalfJung RalfJung mentioned this issue Jul 24, 2019
Centril added a commit to Centril/rust that referenced this issue Jul 25, 2019
…lexcrichton

ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar

This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new `TOOLSTATE_PUBLISH` environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk.

For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load:

```yaml
- bash: echo foo
  env:
    SECRET_VAR: $(SECRET_VAR)
```

This works fine when the variable is present but when it's missing, instead of setting `SECRET_VAR` to an empty string or just not setting it at all, Azure Pipelines puts the literal `$(SECRET_VAR)` as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing:

```yaml
# At the top:
variables:
  - name: MAYBE_SECRET_VAR
    value: $[ variables.MAYBE_SECRET_VAR ]

# In the step:
- bash: echo foo
  env:
    SECRET_VAR: $(MAYBE_SECRET_VAR)
```

While that *could've worked* it was ugly and messy, so I just opted to add yet another non-secret variable.

r? @alexcrichton
fixes rust-lang#62811
Centril added a commit to Centril/rust that referenced this issue Jul 26, 2019
…lexcrichton

ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar

This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new `TOOLSTATE_PUBLISH` environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk.

For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load:

```yaml
- bash: echo foo
  env:
    SECRET_VAR: $(SECRET_VAR)
```

This works fine when the variable is present but when it's missing, instead of setting `SECRET_VAR` to an empty string or just not setting it at all, Azure Pipelines puts the literal `$(SECRET_VAR)` as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing:

```yaml
# At the top:
variables:
  - name: MAYBE_SECRET_VAR
    value: $[ variables.MAYBE_SECRET_VAR ]

# In the step:
- bash: echo foo
  env:
    SECRET_VAR: $(MAYBE_SECRET_VAR)
```

While that *could've worked* it was ugly and messy, so I just opted to add yet another non-secret variable.

r? @alexcrichton
fixes rust-lang#62811
Centril added a commit to Centril/rust that referenced this issue Jul 26, 2019
…lexcrichton

ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar

This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new `TOOLSTATE_PUBLISH` environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk.

For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load:

```yaml
- bash: echo foo
  env:
    SECRET_VAR: $(SECRET_VAR)
```

This works fine when the variable is present but when it's missing, instead of setting `SECRET_VAR` to an empty string or just not setting it at all, Azure Pipelines puts the literal `$(SECRET_VAR)` as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing:

```yaml
# At the top:
variables:
  - name: MAYBE_SECRET_VAR
    value: $[ variables.MAYBE_SECRET_VAR ]

# In the step:
- bash: echo foo
  env:
    SECRET_VAR: $(MAYBE_SECRET_VAR)
```

While that *could've worked* it was ugly and messy, so I just opted to add yet another non-secret variable.

r? @alexcrichton
fixes rust-lang#62811
Centril added a commit to Centril/rust that referenced this issue Jul 26, 2019
…lexcrichton

ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar

This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new `TOOLSTATE_PUBLISH` environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk.

For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load:

```yaml
- bash: echo foo
  env:
    SECRET_VAR: $(SECRET_VAR)
```

This works fine when the variable is present but when it's missing, instead of setting `SECRET_VAR` to an empty string or just not setting it at all, Azure Pipelines puts the literal `$(SECRET_VAR)` as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing:

```yaml
# At the top:
variables:
  - name: MAYBE_SECRET_VAR
    value: $[ variables.MAYBE_SECRET_VAR ]

# In the step:
- bash: echo foo
  env:
    SECRET_VAR: $(MAYBE_SECRET_VAR)
```

While that *could've worked* it was ugly and messy, so I just opted to add yet another non-secret variable.

r? @alexcrichton
fixes rust-lang#62811
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants