upgrade-provider no longer adds Fixes keyword and issue to automatic PRs

[guineveresaenger]

What happened?

See example below - it seems that automation-created issues are missing the "Fixes" keyword.

Locally created PRs as pulumi/pulumi-dbtcloud#168 do not have this problem.

Example

pulumi/pulumi-auth0#687

should have closed

pulumi/pulumi-auth0#686

but was missing the "Fixes <issue #>" text .

Output of pulumi about

n/a

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[danielrbradley]

Ok, making some headway with this today … I’ve created tags in the upstream xyz provider so we can run upgrades on pulumi-xyz without it failing.

I’ve also added the ability to set the version of upgrade-provider when running in CI for better predictability and to enable testing.

Observations:

1. When using CI to create both the issue and PR, I’m still seeing no “fixes” appear.
2. When I re-run the PR creation locally it adds the “fixes”.
3. When closing existing PRs and re-running in CI, I also see the “fixes”.

Therefore, I think we’re running into an issue with a delay in the GitHub issue searching. I think there’s a delay between opening an issue and being able to search on the contents of the body.

I would guess searching on author doesn’t have this issue because it’s using a simpler indexing strategy which is updated synchronously at the point of saving, but the text body searching is only “eventually consistent”.

Therefore, I could see a few approaches to solving this:

1. Add a delay between creating the issue and starting the PR creation (not good IMO)
2. Remove the filtering on pulumiupgradeproviderissue so we just fetch the 100 latest issues in the repo and look at their titles for matching the exact upgrade issue title format
3. Try switching away from a keyword in the body and see if labels indexing is faster. This would require setting up new labels in all repos and reworking more aspects.