migration from trac to Git**b

See also: https://trac.sagemath.org/ticket/30363

Proposed workflow on GitHub (with transition guide from Trac)

• One time action: Instead of depositing an SSH public key on Trac:

  • No action needed if you have already contributed to any other project on GitHub and set up Git credentials or SSH keys for this.
  • For new users of GitHub:
    • Either https://docs.github.com/en/get-started/getting-started-with-git/caching-your-github-credentials-in-git
    • Or generate ssh keypair, or use an already existing one, and upload the public key to your GitHub account settings

• One time action: Instead of adding a git remote named trac:

  • Create your personal GitHub fork of the main repository https://github.com/sagemath/sage - this will become a repository in https://github.com/USERNAME
  • If you already have a clone of a Sage repository on your computer:
    • Check your current git remote repositories:

      git remote -v
      

    • If you already have the main Sage repository (https://github.com/sagemath/sage) as a remote, and its name is not upstream, rename it to upstream using git remote rename OLD-NAME upstream
    • Otherwise, add a new remote:

      git remote add upstream https://github.com/sagemath/sage.git
      

      Alternatively, use ssh access with your ssh keypair - see (Optional) above:

      git remote add upstream [email protected]:sagemath/sage.git
      

    • If you already have a remote named origin and it is not your personal fork, rename this remote to something else using git remote rename origin MY-OLD-ORIGIN
    • Finally, add your fork as a remote via (the URL can be copied from there)

      git remote add origin https://github.com/USERNAME/sage.git
      

      Alternatively, with ssh access (see above):

      git remote add origin [email protected]:USERNAME/sage.git
      

  • Otherwise (fresh start):
    • Clone the forked repository, and do one of the following, depending on the access type (https vs ssh)

      git clone https://github.com/USERNAME/sage.git   # https
      git clone [email protected]:USERNAME/sage.git       # ssh
      

      This will link your fork as the origin remote in the local git.
    • Configure git to sync your fork with the main Sage repository, and do one of the following, depending on the access type (https vs ssh):

      git remote add upstream https://github.com/sagemath/sage.git   # https
      git remote add upstream [email protected]:sagemath/sage.git       # ssh
      

  • (Of course, you can give arbitrary names to your git remotes, but origin and upstream are the established defaults, which will make it easier to use tools such as the GitHub command-line tools.)

• For reporting a bug, planning an enhancement, describing a project, instead of opening a Trac ticket:

  • Open an Issue on GitHub. Preview of Issues (converted from Trac): https://github.com/sagemath/trac_to_gh/issues
  • Trac ticket box attributes are mapped as follows:
    • Type ("defect", "enhancement", "task") is mapped to a "Label" with prefix t:, e.g., t: bug
    • Component ("basic arithmetic", "linear algebra", "geometry", ...) are mapped to "Labels" with prefix c:
    • Priority ("major"/"minor"/"critical") is mapped to "Labels" with prefix p:
    • Keywords can be mapped to "Labels"
    • Cc: use @USERNAME either in the Issue description or in any comment.
      • Optionally, regular developers who would like to get notified automatically when a PR touches a particular part of the sage code can add themselves as a Code Owner.
    • Description becomes just the first comment on the Issue
    • Branch/Commit/Authors/Reviewers/Work Issues: via Pull Requests (PR), see below
    • Report Upstream is replaced by automatic cross references between Issues/PRs in different repos
    • Milestone = duplicate/invalid/wontfix is replaced by marking as duplicate or closing with a comment

• For contributing a change that does not address an existing open Issue, instead of opening a Trac ticket and pushing a git branch to it:

  • Create a new local branch based on upstream/develop
  • Push the branch to the remote named origin, i.e to your fork
  • A git message will provide a URL for opening a Pull Request (PR)
  • Create the PR
  • If it is not ready for review, mark the PR as a "Draft"

• For contributing a change that addresses an existing open Issue, instead of pushing a git branch to a Trac ticket:

  • same as above
  • use Fixes #ISSUENUMBER to link to an existing issue; this will auto-close the linked Issue when the PR is merged.

• For finding PRs that are waiting for review, instead of using Trac ticket reports:

  • filter PRs by review status

• Instead of adding a comment to a ticket:

  • Add a comment to the Issue
  • If a PR is linked to the Issue, you can alternatively comment on the PR.
  • Where should a comment go?
    • To say that the reported issue is not a bug but a feature, the comment should go on the Issue
    • To point out typos in the changes, the comment should go on the PR
    • ... (add your examples)
  • Generally everyone has a large enough screen to view both the Issue and the PR on their screen simultaneously

• For reviewing a change:

  • instead of looking at the patchbot, use the Checks on GitHub Actions, which are already available on Trac since the Sage 9.6 series; the status of the check runs will be clearer on GitHub than on Trac
  • instead of copy-pasting parts of the diff of a branch to a comment, use pull request reviews: You can add comments directly to changed lines
  • instead of making reviewer edits, smaller suggestions can be made through the github web interface as part of the pull request review
  • if you want to be able to make changes directly to others' PRs (when the author elects to allow edits from maintainers), please contact one of the Sagemath Github admins, who can give you the relevant permissions.
  • instead of changing the status of the ticket (i.e. "positive review" or "needs work"), choose the correct type of your pull request review (i.e. "approve" vs "request changes")
  • for trying the branch of a PR locally, instead of using git trac checkout TICKETNUMBER or git trac try TICKETNUMBER, use git fetch upstream pull/PULL_REQUEST_ID/head:LOCAL_BRANCH_NAME
    • alternatively, with the GitHub command-line interface, use gh pr checkout PULL_REQUEST_ID

• For organizing, instead of using meta-tickets:

  • either open an Issue
    • it can include a checklist of things to do which can be checked off as they are dealt with by various PRs.
  • or create a new Project

• Unchanged: Release Manager @vbraun merges positively reviewed tickets into his branch https://github.com/vbraun/sage

  • The release manager uses a filter to identify the pull requests that a reviewer has approved
  • Once released (currently targeted for Q4 2022), we instead use Merge Queues.

• Unchanged: To make a beta or stable release, Release Manager merges (fast-forward) his branch into the develop branch and creates a tag

• Unchanged: To make a stable release, Release Manager merges (fast-forward) the develop branch into the main branch.

  • Only change is the rename of master to main due to cultural sensitivity - as proposed in https://trac.sagemath.org/ticket/31287
  • In the future, we might migrate from this Gitflow workflow to the Trunk-based workflow where the develop branch is no longer needed and changes are directly merged into main.

Proposed permissions and protections

Main repository https://github.com/sagemath/sage:

• Only 2 named branches, develop and master
• everything else goes through PRs
• Create a new team: https://github.com/orgs/sagemath/teams "Release Manager"
• Set up branch protection rule: Only "Release Manager" team can push; no override for users in Admin role; no deletions; no force pushes
• Review/update https://github.com/orgs/sagemath/teams/core/members
• Create a new team for other developers, give Write access to team in repo, add people to team
  • the Write access does not allow pushing to develop or master
  • the Write access allows members of the team to push commits to branches of PRs (unless the PR owner has disabled this for this PR)

Conversion of Trac tickets and the Trac wiki to GitHub

Conversion script: https://github.com/sagemath/trac-to-github

• issues there are various technical discussions on the topic

Preview of the converted issues (iteration 0):

• https://github.com/sagemath/trac_to_gh/issues

To be done: Several iterations of:

• Convert all Trac tickets and Trac wiki to a temporary repo.
• Inspect the result and refine the conversion script.

Switchover day (date to be determined):

1. Make Trac read-only:
   • Take Trac offline,
   • Reconfigure Trac ticket system, Trac wiki to be read-only
   • Make sure Trac git repo is fully synced to https://github.com/sagemath/sagetrac-mirror
   • Make Trac git repo read-only
   • Bring Trac online.
2. Convert all tickets to Issues in a new repo. (This preserves the ticket numbers as Issue numbers.)
3. Open PRs from sagetrac-mirror to the new repo for all open tickets with attached branches.
4. Final check that the new repo is OK.
5. Archive https://github.com/sagemath/sagetrac-mirror (= set to read-only)
6. Replace sagemath/sage by the new repo.
7. Announce that sagemath/sage is now open for Issues and PRs.

Retrieving data from GitHub

GitHub REST API

Particularly, anything extracting/archiving discussions should probably look at Issues API, because "Pull requests are just issues with code", although there is a separate Pull request review comments API. Special care may need to be taken to preserve cross-references when archiving.

3 minute video demoing importing github repos to gitlab, which emphasizes answers to a lot of natural frequent questions (involving users, issue comments, labels, etc.): https://www.youtube.com/watch?v=VYOXuOg9tQI