[Question] Keeps processing same set of issues

[worldomonation]

Hi 👋 first off, thank you for this project - it runs much better than what my org has been using in the past.

I have a question regarding the bot's behavior.

With the following parameters:

• operations-per-run: 30
• ascending: true
• days-before-issue-stale: 180
• days-before-close: -1
• exempt-issue-labels: '[Pri] High,[Pri] BLOCKER,[Status] Keep Open'

In the first week or so of execution the bot managed to identify and apply the stale label to a good number of issues.
However, for the last few days the bot appears to be processing the same set of issues, starting from oldest open issue on file.

The old issues get repeatedly processed are actually alive or recently had the stale label applied.

My question would be:
a) is there a way to exclude the previously processed issues from re-processing in the subsequent days?
and
b) if the above behavior cannot be accomplished with the bot as-is, would increasing the operations-per-run to a much higher value eg. 2000 be enough to deal with the issue?

The total issues for the particular repository number about ~2300 and while a good chunk of them are stale, a lot of new issues continue to pile in so realistically only about 1/2 meet the s>180 days without activity threshold.

[C0ZEN]

@worldomonation

is there a way to exclude the previously processed issues from re-processing in the subsequent days?

Nope and I am not sure that this is a good idea to filter any issue/PR because it could cause unexpected behaviour with the stale workflow.

if the above behavior cannot be accomplished with the bot as-is, would increasing the operations-per-run to a much higher value eg. 2000 be enough to deal with the issue?

Yes, but at the cost to have GitHub API rate limits to be reached causing issues.

If you have the time, you could change the configuration to enable the debug and explore the logs.
You will be able to know what actually is happening under the hook.
You can read this small section to know how.

On the other hand, I really think that this issue with the API is pretty inconvenient for everyone and the option is often mislead.

I see a room for improvement by either:

• Find a good way to know exactly what is the current rate limit
• Adding a new "delay" option to process the issues but at a slower rate but I do know at all if this would help to avoid reaching the limit and if it's a good thing to have anyway because when processing a huge project it can be very annoying to have and likewise when debugging and so on

I saw that GitHub provide a way to know this information and in that case I am wondering if we would just call it, get the remaining and change the operations-per-run option to be instead a limit of operations only applied to avoid consuming all the remaining rate limit.
Eventually, @hross do you have some insights about this?

[worldomonation]

Hi @C0ZEN thanks for your insight!

If you have the time, you could change the configuration to enable the debug and explore the logs.
You will be able to know what actually is happening under the hook.
You can read this small section to know how.

If my understanding is correct, setting the following:

• operations-per-run: 30
• debug-only: true

Would not incur actual API calls - is this correct?

* Adding a new "delay" option to process the issues but at a slower rate but I do know at all if this would help to avoid reaching the limit and if it's a good thing to have anyway because when processing a huge project it can be very annoying to have and likewise when debugging and so on

I think this behavior would be a great option to have for projects that have a large number of issues that cannot be automatically closed (for a variety of reasons).

In any case, I will play around wit the debug mode some more and hopefully can come to a solution that works for my project. Thanks!

[C0ZEN]

@worldomonation yes, debug-only is a dry-run action so the GitHub API will not be called (except to fetch the issues and PRs) so you can play with the options without impacting your project.

ACTIONS_STEP_DEBUG is used on the other hand to display the log in the workflow.
You could try to enable it and see what's what with your current configuration and then if you want to play with the options you can use debug-only to avoid any issue.
Note that the workflow will keep counting the number of operations so this is a good way to know if increasing the limit could help or not.

[C0ZEN]

@worldomonation any update? In which project are you using it?

[worldomonation]

@worldomonation any update?

Sorry about that!

In which project are you using it?

I am using this Action in Automattic/wp-calypso repository. There are ~2300 issues at this time, with a good chunk of it stale.

I've tweaked the parameters somewhat to make sure it can catch most of the stale issues, starting from oldest first. I found ~500 actions per run to process enough issues, but because closing issues is done manually in the repository, new issues being marked as stale has slowed to a trickle.

[C0ZEN]

@worldomonation you have probot to stale and this action 😲
I am curious about the two living together!

[worldomonation]

@worldomonation you have probot to stale and this action 😲
I am curious about the two living together!

probot/stale is what we used to use, but I have deprecated it in favor of this action since probot/stale did not seem to do anything for some time. As soon as I enabled this action, it started marking issues that probot/stale had not done anything on for over half a year.

[C0ZEN]

@worldomonation I will provide a small PR to improve a little bit the logs to have a better understanding of what is going on for consumers so they don't need to dive in the code and it's always helpful anyway.

Regarding this part especially:

if (!issue.isStale && shouldBeStale && shouldMarkAsStale) {
  issueLogger.info(
    `Marking $$type stale because it was last updated on ${issue.updated_at} and it does not have a stale label`
  );
  await this._markStale(issue, staleMessage, staleLabel, skipMessage);
  issue.isStale = true; // this issue is now considered stale
} else if (!issue.isStale) {
  issueLogger.info(
    `Not marking as stale: shouldBeStale=${shouldBeStale}, shouldMarkAsStale=${shouldMarkAsStale}`
  );
}

Also I think that it can be good to have some logs regarding the consumption of the infamous "operations per run" for each issue.

It gave me once again a reason to improve this option and I saw the other day that potentially we could fetch this information from the GitHub API to know exactly how many operations are possible so maybe providing a small PR just to log the result of this endpoint can be a good lead for the next improvements.

Finally, if you are not aware soon there will be a new release and it will include some statistics at the end of the logs to sums up basically the mains actions (counter form).
It can give you some good insights of what is going on for each run of the action.
Plus a small improvement regarding the logs to make them a little bit more human friendly.

I am very interested to see that in the action for such a huge project.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[C0ZEN]

up

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[C0ZEN]

up

[C0ZEN]

@worldomonation FYI it was released in https://github.com/actions/stale/releases/tag/v3.0.19

[worldomonation]

@worldomonation FYI it was released in https://github.com/actions/stale/releases/tag/v3.0.19

I appreciate your continued work on this!

I must say, barring my own mistake (accidentally un-labeling all issues that stale marked), I am finding stale to work quite well. Of the ~2.4k issues and ~280 PRs we have open currently, stale seems to get through all of it without issue.

[C0ZEN]

@worldomonation nice to hear! If you want to be sure that the stale action process thoroughly everything you can just check for the last line of the script.
If there is a warning suggesting increasing the https://github.com/actions/stale#operations-per-run then just increase the limit or keep it like this if you think that this is still enough.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[C0ZEN]

up

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[C0ZEN]

@worldomonation do you think it can be closed now?

[worldomonation]

@worldomonation do you think it can be closed now?

I think we can close this. Thanks for all your support on this!

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days