Edits to contributors-data.js and schedule-monthly.yml

[t-will-gillis]

Dependents

• Bug fix and minor edits get-timeline.js #6459
• Create new file inactive-members.json #6460
• Edits to create-new-issue.js  #6396

Associated Files

• Updates to inactive-members.md file #6395

Overview

The current contributors-data.js has become unwieldly and much of its functionality can be modularized. There are also several additional edits that we can make so that these files are more compatible and exchangeable with other workflows, and there are other checks and features that should be added to the inactive members workflow as detailed following.

Details/ Main Objectives

• Replace octokit/rest object with actions/github-script@v7 to make more compatible with other workflows
• Move contributors-data.js to same folder as second half of workflow so files are in same folder (ie. into /list-inactive-members/
• Refactor/ modularize contributors-data.js into more discrete functionalities
• Extract getTimeline() to the common /utils/ folder
• Extract getTeamMembers() to the common /utils/ folder
• Add new function to close an inactive member's 'Pre-work Checklist' if it is open
• In the function that checks whether an Inactive member is new: Replace the existing check with a more accurate check (looks for a repo with a description matching HfLA) and remove the check for whether the member is a "Maintainer".
• Add functionality to confirm that inactive members have already received prior warning that they were about to be removed if they remain inactive- (For various reasons, some members might be on the 'to remove' list without being placed on the 'to notify' list in the previous month- one example is an inactive member who recently closes their Pre-Work Checklist)

Action Items

• Move and rename /github-data/contributors-data.js to list-inactive-members/get-contributors-data.js
  For the next change, refactor from the use of the octokit object to using the github context generated by actions/github-script. In this way, this workflow can more readily make use of, and contribute to, the reusable utility functions in the github-actions/utils/ folder.

• At the beginning of the file, replace the Octokit object:

  const fs = require("fs");
  const { Octokit } = require("@octokit/rest");
  
  // Extend Octokit with new contributor endpoints and construct instance of class with Auth token 
  Object.assign(Octokit.prototype);
  const octokit = new Octokit({ auth: process.env.token });
  
  // Set variables to avoid hard-coding
  const org = 'hackforla';
  const repo = 'website';
  const team = 'website-write';
  const baseTeam = 'website';
  const maintTeam = 'website-maintain';
  

  with:

  // Import modules
  const fs = require('fs');
  const getTimeline = require('../../utils/get-timeline');
  const getTeamMembers = require('../../utils/get-team-members');
  
  // Global variables
  var github;
  var context;
  
  // Set variables to avoid hard-coding
  const org = context.repo.owner;
  const repo = context.repo.repo;
  const baseTeam = 'website';
  const writeTeam = 'website-write';
  const mergeTeam = 'website-merge';
  // const adminTeam = 'website-admins';   <-- Not used currently
  const maintTeam = 'website-maintain';
  

• Replace all references to octokit with github

• Replace team with writeTeam, around 5 locations.

• Copy this file to list-inactive-members/trim-inactive-members.js

• At the main() function declaration, replace:

  /**
   * Main function, immediately invoked
   */
  (async function main(){ 
    const [contributorsOneMonthAgo, contributorsTwoMonthsAgo, inactiveWithOpenIssue] = await fetchContributors();
    console.log('-------------------------------------------------------');
    console.log('List of active contributors since ' + oneMonthAgo.slice(0, 10) + ':');
    console.log(contributorsOneMonthAgo);
  
    const currentTeamMembers = await fetchTeamMembers(team);
    console.log('-------------------------------------------------------');
    console.log('Current members of ' + team + ':')
    console.log(currentTeamMembers)
  
    const [removedContributors, cannotRemoveYet] = await removeInactiveMembers(currentTeamMembers, contributorsTwoMonthsAgo, inactiveWithOpenIssue);
    console.log('-------------------------------------------------------');
    console.log('Removed members from ' + team + ' inactive since ' + twoMonthsAgo.slice(0, 10) + ':');
    console.log(removedContributors);
  
    console.log('-------------------------------------------------------');
    console.log('Members inactive since ' + twoMonthsAgo.slice(0, 10) + ' with open issues preventing removal:');
    console.log(cannotRemoveYet);
    
    const updatedTeamMembers = await fetchTeamMembers(team);
    const notifiedContributors = await notifyInactiveMembers(updatedTeamMembers, contributorsOneMonthAgo);
    console.log('-------------------------------------------------------');
    console.log('Notified members from ' + team + ' inactive since ' + oneMonthAgo.slice(0, 10) + ':');
    console.log(notifiedContributors);
  
    writeData(removedContributors, notifiedContributors);
  })();  
  
  

  with:

  /**
   * Main function
   * @param {Object} g          - github object from actions/github-script
   * @param {Object} c          - context object from actions/github-script
   * @return {Object} results   - object to continue in `trim-inactive-members.js`
   */
  async function main({ g, c }) {
    github = g;
    context = c;  
  
    const [contributorsOneMonthAgo, contributorsTwoMonthsAgo, inactiveWithOpenIssue] = await fetchContributors(dates);
    console.log('-------------------------------------------------------');
    console.log('List of active contributors since ' + dates[0].slice(0, 10) + ':');
    console.log(contributorsOneMonthAgo);
  
    let results = {};
    results["recentContributors"] = contributorsOneMonthAgo;
    results["previousContributors"] = contributorsTwoMonthsAgo;
    results["inactiveWithOpenIssue"] = inactiveWithOpenIssue;
    results["dates"] = dates;
    
    return results;
    };
  
  

• Remove the function getEventTimeline(issueNum) in its entirety, and replace:

  const timeline = await getEventTimeline(issueNum);
  

  with:

  const timeline = await getTimeline(issueNum, github, context);
  

• Move function fetchTeamMembers() to new file utils/get-team-members.js

• Remove all code from removeInactiveMembers() to end and add line module.exports = main

In the new trim-inactive-members.js:

• Remove the section defining dates, it is not needed here

• At the main() function, replace with edited section:

    /**
   * Main function
   * @param {Object} g        - github object from actions/github-script
   * @param {Object} c        - context object from actions/github-script
   * @param {Object} results  - results from `get-contributors-data.js`
   */
  async function main({ g, c }, results) {
    github = g;
    context = c;
  
    const recentContributors = results["recentContributors"];                                 // recentContributors per the dates[0] cutoff
    const previousContributors = results["previousContributors"];                             // previousContributors per the dates[1] cutoff
    const inactiveWithOpenIssue = results["inactiveWithOpenIssue"];
    const dates = results["dates"];
    
    const currentTeamMembers = await getTeamMembers(github, context, writeTeam);
    console.log('-------------------------------------------------------');
    console.log('Current members of ' + writeTeam + ':');
    console.log(currentTeamMembers);
  
    const [removedContributors, cannotRemoveYet] = await removeInactiveMembers(previousContributors, inactiveWithOpenIssue);
    console.log('-------------------------------------------------------');
    console.log('Removed members from ' + writeTeam + ' inactive since ' + dates[1].slice(0, 10) + ':');
    console.log(removedContributors);
  
    console.log('-------------------------------------------------------');
    console.log('Members inactive since ' + dates[1].slice(0, 10) + ' with open issues preventing removal:');
    console.log(cannotRemoveYet);
  
    // Repeat getTeamMembers() after removedContributors to compare with recentContributors
    const updatedTeamMembers = await getTeamMembers(github, context, writeTeam);
    const notifiedContributors = await notifyInactiveMembers(updatedTeamMembers, recentContributors);
    console.log('-------------------------------------------------------');
    console.log('Notified members from ' + writeTeam + ' inactive since ' + dates[0].slice(0, 10) + ':');
    console.log(notifiedContributors);
  
    writeData(removedContributors, notifiedContributors, cannotRemoveYet);
  };  
  
  

• Remove fetchContributors() through isEventOutdated()

• Within the function removeInactiveMembers(), add functionality to confirm that a member to-be-removed received notification in the prior month, and remove member from 'website-merge' if needed, and finally close a removed member's "Pre-work Checklist" if open. Replace:

  // Remove member from the team if they don't pass additional checks in `shouldRemoveOrNotify` function
        if(await shouldRemoveOrNotify(username)){
          // But if member has an open issue, don't remove
          if(username in inactiveWithOpenIssue){
            cannotRemoveYet[username] = inactiveWithOpenIssue[username];
          } else {
            await octokit.request('DELETE /orgs/{org}/teams/{team_slug}/memberships/{username}', {
              org: org,
              team_slug: team,
              username: username,
            })
            removedMembers.push(username);
          }
        } 
      }
  

  with:

        // Confirm that member is  not new/ did not create HfLA repo clone in last month
        if(await checkMemberIsNotNew(username)){
          // But if member has an open issue or was not on the previously notified list, do not remove yet
          if(username in inactiveWithOpenIssue && inactiveWithOpenIssue[username][1] === false){
            cannotRemoveYet[username] = inactiveWithOpenIssue[username][0];
          } else if((previouslyNotifed.length > 0) && !(previouslyNotified.includes(username))){
            console.log('Member was not on last month\'s \'Inactive Members\' list, do not remove: ' + username);
          } else {
            // Remove member from all teams (except baseTeam)
            const teams = [writeTeam, mergeTeam];
            for(const team of teams){
              await github.request('DELETE /orgs/{org}/teams/{team_slug}/memberships/{username}', {
                org: org,
                team_slug: team,
                username: username,
              });
            }
            removedMembers.push(username);
            // After removal, close member's "Pre-work checklist" if open
            if(username in inactiveWithOpenIssue && inactiveWithOpenIssue[username][1] === true){
              closePrework(username, inactiveWithOpenIssue[username][0]);
            }
          }
        } 
      }
  

• Add function to automatically close an inactive member's Pre-work:

    /**
   * Function to close a just-removed inactive member's "Pre-work checklist" if open and add a comment
   * @param {String} member        - name of member whose "Pre-work checklist" will be closed
   * @param {Number} issueNum      - number of member's "Pre-work checklist"
   */
  async function closePrework(member, issueNum){ 
    // Close the assignee's "Pre-work Checklist" and add comment
    await github.request('PATCH /repos/{owner}/{repo}/issues/{issue_number}', {
      owner: org,
      repo: repo,
      issue_number: issueNum,
      state: 'closed'
    });
    console.log('Closing "Pre-work Checklist" issue number ' + issueNum + ' for ' + member);
    // Add comment to issue
    await github.request('POST /repos/{owner}/{repo}/issues/{issue_number}/comments', {
      owner: org,
      repo: repo,
      issue_number: issueNum,
      body: 'The Hack for LA Bot has closed this issue due to member inactivity.'
    });
  }  
  
  

• Remove the entirety of function shouldRemoveOrNotify() since we are not checking whether a member is a Maintainer when determining if the member is Inactive. Replace with a new check for whether a member has recently cloned the HfLA repo, since we don't want to remove members that signed up recently:

    /**
   * Function to confirm that the member did not recently clone HfLA repo and thus is not new: if so , otherwise they are new
   * @param {String} member - Member's username 
   * @returns {Boolean}           - True if member is not new, False if member is new
   */
  async function checkMemberIsNotNew(member){
    // Check if member cloned the HfLA repo within the last 30 days. If so, they are new members and they
    // may not have had time yet to get set up. Check the user's 10 most recent repos and see if any match 
    // "description":"Hack for LA's Website": if so then, check if "created_at" is less than 30 days.
    try {
      const memberRepoResults = await github.request('GET /users/{username}/repos', {
        username: member,
        direction: 'asc',
        sort: 'created',
        per_page: 10,
      });
      for(const memberRepo of memberRepoResults.data){
        if(memberRepo.description === "Hack for LA's website" && memberRepo.created_at > oneMonthAgo){
          console.log("Member created organization repo within last month: " + member);
          return false;
        }
      }
    } catch {}
  
    // Else this member is not new and should be notified or removed from team as appropriate
    return true;
  }
  

• Add function that will read the json file for the previous month's data to confirm that this month's to-be-removed members received a notification the prior month:

  
  /**
   * Function to find the previous month's "Review Inactive Team Members" issue and extract the raw notified members list
   * @params {}              - none
   * @returns {Array}        - list of notified members (f. prev. month) 
   */
   async function readPreviousNotifyList(){
  
     try {
       // Retrieve previous month's inactive member list 
       const filepath = 'github-actions/utils/_data/inactive-members.json';
       const rawData = fs.readFileSync(filepath, 'utf8');
       const parsedData = JSON.parse(rawData);
       const notifiedMembers = parsedData['notifiedContributors'];
  
       return notifiedMembers;  
  
       // If error, return empty array
     } catch (err) {
       throw new Error(err);
       return [];
     }
  } 
  

• Finally, add module.exports = main at end

• In the schedule-monthly.yml file we will be using actions/github-script@v7, thus we do not need the steps to setup Node or install the npm dependencies. Also note that the standard HfLA token is replaced by a token with extra ADMIN privileges to allow the workflow to run correctly. Replace:

      # Setup node 
    - name: Setup node
      uses: actions/setup-node@v4
      with:
        node-version: 18
        cache: 'npm'
  
    # Install dependencies to run js file
    - name: Install npm dependencies
      run: npm install
      working-directory: ./github-actions/trigger-schedule/github-data
  
    # Run js file: checks contributor activity logs, removes two-month inactive members from 
    # 'website-write' team, then compiles list of one-month inactive members for notification
    - name: Trim Members
      env:
        token: ${{ secrets.HACKFORLA_BOT_PA_TOKEN }}
      run: node github-actions/trigger-schedule/github-data/contributors-data.js
  

  with:

      # Checks member activity logs for recent and previous contributors 
    - name: Get Contributors
      id: get-contributors
      uses: actions/github-script@v7
      with:
        github-token: ${{ secrets.HACKFORLA_ADMIN_TOKEN }}
        script: |
          const script = require('./github-actions/trigger-schedule/list-inactive-members/get-contributors-data.js')
          const results = script({ g: github, c: context })
          return results
  
    # Trims inactive members from team and notifies idle members
    - name: Trim and Notify Members
      id: trim-and-notify-members
      uses: actions/github-script@v7
      with:
        github-token: ${{ secrets.HACKFORLA_ADMIN_TOKEN }}
        script: |
          const results = ${{ steps.get-contributors.outputs.result }}
          const script = require('./github-actions/trigger-schedule/list-inactive-members/trim-inactive-members.js')
          script({ g: github, c: context }, results)
  

• Remove steps to save artifact, and instead save json data to repo. Replace:

     # Upload artifact file to allow list sharing with next job "Create_New_Issue"
    - name: Upload artifact
      uses: actions/upload-artifact@v4
      with:
        name: trim_job_artifact
        path: inactive-Members.json 
  
  Create_New_Issue:
    needs: Trim_Contributors
    runs-on: ubuntu-latest     
    steps:
    - uses: actions/checkout@v4
  
    # Download artifact file from "Trim_Contributors"
    - name: Download artifact
      id: download-artifact
      uses: actions/download-artifact@v4
      with:
        name: trim_job_artifact
  
    # Extract and save artifact in usable form for next steps
    - name: Extract artifact
      id: extract-artifact
      run: |
        jq -c . inactive-Members.json > out-inactive-Members.json
        echo "TRIM_LISTS=$(cat out-inactive-Members.json)" >> $GITHUB_ENV
  

  with:

      # Commits list of inactive & removed members to repo for immediate and next month's use
    - name: Update Inactive Members JSON
      uses: stefanzweifel/[email protected]
      with:
       # Glob pattern of file which should be added to the commit
        file_pattern: github-actions/utils/_data/inactive-members.json
  
        # Optional commit message and author settings
        commit_message: Update Inactive Members JSON
        commit_author: GitHub Actions Bot <[email protected]> 
  
  Create_New_Issue:
    needs: Trim_Contributors
    runs-on: ubuntu-latest     
    steps:
    - uses: actions/checkout@v4
  

Resources/Instructions

schedule-monthly.yml
contributors-data.js

[t-will-gillis]

This issue has exploded in complexity. I split out a couple smaller issues (#6395, #6396, #6459, #6460) to help bring the scope down.

• Updates to inactive-members.md file #6395
• Edits to create-new-issue.js  #6396
• Bug fix and minor edits get-timeline.js #6459
• Create new file inactive-members.json #6460

[t-will-gillis]

Workflow ready to be implemented.

[ExperimentsInHonesty]

@t-will-gillis Under resources, it says

Related issues:

schedule-monthly.yml
contributors-data.js
create-new-issue.js
inactive-members.md

Is this the epic, or is there an epic that this all connects to?

[t-will-gillis]

@ExperimentsInHonesty Under Resources, I removed the references to the other files and left the two that are being edited. There is not an epic that this connects to so this issue could be the epic, or I could create a separate epic if there should be one. However you think is best-

My mindset was that originally this issue would address everything, but then I split off the other issues since they can be edited mostly independently from this one, and this one was already very large by itself. I was planning to release a PR for this issue once the other four are done.

[t-will-gillis]

@ExperimentsInHonesty Also, if it makes a difference, I am planning to submit the PR for this one assuming you approve of the changes.

[t-will-gillis]

Hi @ExperimentsInHonesty I added the ready for product label in case you want to review this and/or edit anything. If the overall objectives look okay to you, I will post a PR.