Github Event Processor tool for issues/pull request automation (#5607)

* Initial Commit of GitHub Actions Processor

* Add ci.yml for building/publishing

* fix error in ci.yml

* Comment out a test that's only failing on Mac

* set packaging for GitHubEventProcessor

* set tool command name to github-event-processor

* Updates for testing and addition of Search rate limit output

* Changes for Octkit 5.x and update Octokit version to 5.02

* Minor tweak to the error message for when the config cannot be found

* Update the retry loop to try 5x and use Task.Delay instead of Thread.Sleep

* Update root level README.md and RULES.md as per feedback

* Building from source, use actions checkout which simplifies things and doesn't require source be in the test repository

* Add testing README and snapshots

* Updates for review feedback

* Update InitialIssueTriage function to add issue tracking link for TBD CODEOWNER updates

* One last update for existing review feedback

tools/github-event-processor/Azure.Sdk.Tools.GitHubEventProcessor.Tests/Azure.Sdk.Tools.GitHubEventProcessor.Tests.csproj

@@ -0,0 +1,34 @@
+<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <TargetFramework>net6.0</TargetFramework>
+    <ImplicitUsings>disable</ImplicitUsings>
+    <Nullable>enable</Nullable>
+
+    <IsPackable>false</IsPackable>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.4.1" />
+    <PackageReference Include="coverlet.collector" Version="3.2.0">
+      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
+      <PrivateAssets>all</PrivateAssets>
+    </PackageReference>
+    <PackageReference Include="NUnit" Version="3.13.3" />
+    <PackageReference Include="NUnit3TestAdapter" Version="4.3.1" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <ProjectReference Include="..\Azure.Sdk.Tools.GitHubEventProcessor\Azure.Sdk.Tools.GitHubEventProcessor.csproj" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <None Update="Tests.JsonEventPayloads\*">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </None>
+    <None Update="Tests.FakeCodeowners\*">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </None>
+  </ItemGroup>
+
+</Project>

tools/github-event-processor/Azure.Sdk.Tools.GitHubEventProcessor.Tests/ProcessingTestBase.cs

@@ -0,0 +1,27 @@
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Text;
+using System.Threading.Tasks;
+using Azure.Sdk.Tools.GitHubEventProcessor.Constants;
+using NUnit.Framework;
+using Octokit.Internal;
+
+namespace Azure.Sdk.Tools.GitHubEventProcessor.Tests
+{
+    /// <summary>
+    /// Testing base class. Each static test is going to be using Octokit's SimpleJsonSerializer to deserialize
+    /// the json payload.
+    /// </summary>
+    public class ProcessingTestBase
+    {
+        // SimpleJsonSerializer is OctoKit's serializer used for deserializing GitHub action payloads
+        SimpleJsonSerializer _serializer = new SimpleJsonSerializer();
+
+        public SimpleJsonSerializer SimpleJsonSerializer
+        {
+            get => _serializer;
+        }
+    }
+}

tools/github-event-processor/Azure.Sdk.Tools.GitHubEventProcessor.Tests/README.md

@@ -0,0 +1,95 @@
+# GitHub Event Processor Tests
+
+NUnit test project for GitHubEventProcessor. It contains a suite of static tests used to test the individual rules that can be run within Visual Studio.
+
+## Why are there only automated Static tests and no automated Live tests?
+
+1. Actions requires something happening with an Issue, Pull_Request, Pull_Request_Review or Comment that's done through through the UI. Creating or modifying things authenticated using a token is not going to cause other events to fire. For example, adding a label to an issue, will not cause an **Issues Labeled** event to fire off.
+2. Actions are repository specific. Even if we could do automated live testing, it would only able to be done a repository setup specifically for it which would not be the same repository that's building, packaging and releasing the tools package.
+3. Rule specific reasons. There are rule specific reasons that would also make this challenging. For example, we have a rules that look for issues or pull_requests that haven't had activity for 7/14/60/90 days and those can't be created on the fly and as soon as one is modified by rules process, that activity counter resets.
+4. Live testing really only ends up testing the GitHub API. Calling to get permissions for a user, checking a user's org, updating an Issue or Pull_Request, or adding a comment, none of these are really going to test anything related to the rules themselves that couldn't be tested with a static payload and mocked calls.
+
+## Static Testing and NUnit
+
+The individual action and scheduled event functions are tested using NUnit and static payloads. The tests are all located in the [Static](./Static/) subdirectory and their GitHub Event Payloads are located in the [Tests.JsonEventPayloads](./Tests.JsonEventPayloads/) subdirectory. Like the event processing code, the tests are divided up by the events that trigger particular workflows and each rule has its own test function. For example, all of the Pull Request processing tests are located in [PullRequestProcessingTests.cs](./Static/PullRequestCommentProcessingTests.cs).
+
+Each rule has at least two tests. The first test should ensure nothing happens if the rule is turned off. The second, and any subsequent tests, run with the rule turned on. These verify that things process as expected given a payload that contains the correct action and criteria for the event. For example, **Pull Request Triage** is rule that processes on a *pull request* event's *opened* action if there the pull request is created with no labels. The processing in the rule will add labels based upon file paths in the PR and will create a comment if the user who opened the PR does not have write or admin collaborator permissions. The permission check, along with anything else that requires a call to GitHub, are mocked for static tests. There are three tests for this particular rule, with the last two scenarios checking behavior with and without admin permissions. While this rule does not, some rules can require multiple payloads to test.
+
+### Generating the static payloads
+
+In order to run actions the following need to be setup:
+
+1. The repository settings need to enable actions. *This can be done on a fork.*
+2. A workflow yml file needs to be in the .github/workflows directory with the appropriate event and action to test. The following example just sets up to output the event payload **issue_comment** event's **created** action and just echos out the event payload.
+3. After the yml file has been checked in, whatever setup required for the payload being generated needs to be done. In this example an issue comment needs to be created. The issue, itself, could be one that already exists but, if not, it would need to be created first. Other events might require more involved setup depending on the event, action and criteria of the rule.
+
+The yml below was put into a file named issue-comment-handler.yml and checked into the .github/workflows directory of the [azure-sdk/github-event-processor-test](https://github.com/azure-sdk/github-event-processor-test) repository.
+
+```yml
+name: Issue Comment Event Handler
+on:
+  issue_comment:
+    types: [created]
+jobs:
+  event-handler:
+    name: Handle ${{ github.event_name }} ${{ github.event.action }} event
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Echo event payload
+        run: |
+          echo $GITHUB_PAYLOAD
+        shell: bash
+        env:
+          GITHUB_PAYLOAD: ${{ toJson(github.event) }}
+```
+
+Once the workflow yml file has been checked into the repository or branch a comment created on an Issue or Pull Request will cause the event to fire. Select Actions in the repository which bring up all the Actions for a given repository. Over on the left, is the list of Workflows and the name, in the yml, should be one of them. Selecting that workflow will only show actions for that particular workflow.
+
+![image](./documentation/images/action_event.png)
+
+Select action event to look at the details for its workflow run.
+
+![image](./documentation/images/action_event_details.png)
+
+The workflow run will show all of the steps from setup, execution and cleanup. The one that matters is the **Echo event payload** which is the name of step that is outputting the payload.
+
+![image](./documentation/images/event_payload.png)
+
+The payload in between the begin/end tags and is usually several hundred lines long. Copy this and save it a file. At this point it's almost ready to be used for static testing, it just needs to be sanitized.
+
+#### Sanitizing the static payloads
+
+User and repository names should not be in the static payload files and sanitizing them requires a few, easy, global replacements.
+
+- replace: azure-sdk/github-event-processor-test
+- with: Azure/azure-sdk-fake
+
+- replace: Actual user name(s)
+- with: FakeUser1 ..FakeUserN
+
+- replace: github-event-processor-test (there's always one "name" field in the repository section that has "github-event-processor-test" and not full name "azure-sdk/github-event-processor-test")
+- with: azure-sdk-fake
+
+#### Scheduled event tests only require a single static payload
+
+Scheduled events differ from Action events in that the payload is vastly simpler. They really only contains the repository information necessary to search issues in GitHub and this information isn't required for the Scheduled event static tests.
+
+### Static tests use a mock GitHubEventClient
+
+The GitHubEventClient is a singleton used by workflow processing and any/all external calls are made through this class as well as calls to load rules configuration and CODEOWNERS files. The [MockGitHubEventClient](./MockGitHubEventClient.cs) overrides all of these calls, setting return values used for testing, or loading files crafted to help test certain scenarios in the case of CODEOWNERS. It's also used to expose things, for testing purposes, that aren't exposed in the GitHubEventClient. For example, the lists of items created by Scheduled events aren't exposed in GitHubEventClient because they don't to be.
+
+## Live Testing
+
+[azure-sdk/github-event-processor-test](https://github.com/azure-sdk/github-event-processor-test) is a repository which was specifically setup for testing purposes and can be used to generate static payloads. The .github\workflow contains  the [config and workflow yml files](../YmlAndConfigFiles/) which, aside from different config files, the final version of the workflow files. Every Action rule can be setup through creation and modification of issues and pull requests within the repository and every action has been tested this way. Scheduled events are a little different to test live because constraints dealing with time. Every scheduled event has "last modified X days ago" as part of its criteria ranging from 7 to 90 days which isn't something that can be created on the fly.
+
+### Other requirements of the test repository
+
+- Actions are turned on
+- Every Azure SDK repository has a set of common labels that are used in rules automation. Those have already been created in this repository. The full list can be found in [LabelConstants.cs](../Azure.Sdk.Tools.GitHubEventProcessor/Constants/LabelConstants.cs) in the main project.
+- Several testing SDK/ServiceDirectories with simplistic contents for creating PRs
+- CODEOWNERS file that's been simplified for testing and tailored to the testing SDK/ServiceDirectories in the repository.
+
+### Live testing using the installed package vs live testing building from source
+
+The default in the workflow yml files is to install the github-event-processor from the [feed](https://dev.azure.com/azure-sdk/public/_artifacts/feed/azure-sdk-for-net/NuGet/Azure.Sdk.Tools.GitHubEventProcessor) which is produced from the Azure/azure-sdk-tools repository, where the source code lives. When doing active development, it's beneficial to be able to test changes live in the repository. Both workflow yml files have been setup to flip between running the installed package from the feed or building and installing the github-event-processor from Azure/azure-sdk-tools at a specific PR or SHA. This is trivial to do and instructions on how to do it are in both workflow yml files. Outside of the test repository, only the installed package should be ever be used. The reason for this is that the test repository does not have the same traffic as other repositories and building downloads several additional package from nuget.