Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect when the default branch is renamed (e.g. from master to main) #7907

Closed
reaperhulk opened this issue Feb 13, 2021 · 25 comments · Fixed by #8988
Closed

Detect when the default branch is renamed (e.g. from master to main) #7907

reaperhulk opened this issue Feb 13, 2021 · 25 comments · Fixed by #8988
Labels
Accepted Accepted issue on our roadmap Support Support question

Comments

@reaperhulk
Copy link

Details

RTD, by default, attempts to guess what the default branch is based on the VCS you have configured. For git it defaults to assuming the name master. With GitHub migrating to main (see: https://github.com/github/renaming) this results in a build failure until the user notices and manually sets the branch to main in the advanced settings.

Expected Result

RTD should keep building after the default branch name is changed to main.

Actual Result

It fails and users need to manually set it in the advanced settings.

@humitos
Copy link
Member

humitos commented Feb 15, 2021

@reaperhulk can you describe what steps you followed?

If you import your project by using our wizard ("Import a Project" button), main being the default branch should have been detected automatically.

If you has a project already imported and you rename master to main, we don't have a way to know that and I'm not sure that we can make that assumption.

@reaperhulk
Copy link
Author

The latter (rename) is what occurred. If RTD can’t detect arbitrary branch rename (although has anyone reported an issue to GH about that?) then in a case where master disappears it seems worthwhile to probe for main and use that if present. It’s possible this will be wrong but it will be right far more often.

@stsewd
Copy link
Member

stsewd commented Feb 15, 2021

Here we aren't assuming the name of the branch, it's the name already given to the latest version, it is hard to us to assume the branch was re-named and not deleted, we are stopping from "doing magic" in favor of users to be explicit. So, I don't think we would implement some custom logic for this case. You can use the API to automate this https://docs.readthedocs.io/en/stable/api/v3.html#project-update ("default_branch").

@stsewd stsewd added the Support Support question label Feb 15, 2021
@pllim
Copy link
Contributor

pllim commented Mar 20, 2021

I have a question related to this. In my case, I actually renamed master to main (not creating a duplicate and also keeping master). In the RTD admin panel, it still says latest is grabbing from master and have an option for me to activate main separately. But in my settings, I have latest set to ----, which says, "What branch 'latest' points to. Leave empty to use the default value for your VCS (eg. trunk or master)." But the default (by which I mean the value GitHub API returns when you ask for default branch) is now "main." But RTD is not reporting failure. What is really going on here? I am confused.

@this-josh
Copy link

I too have a question relating to this. My packages main branch is main, it has never had a master branch. When I first setup the project in RTD it chooses the branch main and successfully builds the docs, however, after the first build RTD changes to use the master branch causing the error error: pathspec 'master' did not match any file(s) known to git.. When I go into advanced settings to change the branch the only option available is ----------. Is it possible for me too explicitly specify a branch for RTD to use?

@stsewd
Copy link
Member

stsewd commented Mar 22, 2021

@pllim @this-josh when the option is empty (------) it defaults to "master" not to the default branch set in GitHub. If there is another problem, please link me to your project so I can take a look.

@this-josh
Copy link

@stsewd. My project: https://github.com/this-josh/felling as you can see there isn't a "master" branch, and in RTD the only option is ------- which is defaulting to "master" so I cannot get my docs to build.

@stsewd
Copy link
Member

stsewd commented Mar 22, 2021

@this-josh can you link me to your Read the Docs project? (not the repo)

@this-josh
Copy link

this-josh commented Mar 22, 2021

@stsewd Sorry, here it is: https://felling.readthedocs.io/en/latest/ . Note it built when I first made the project as it correctly identified the main branch.

@pllim
Copy link
Contributor

pllim commented Mar 22, 2021

it defaults to "master" not to the default branch set in GitHub

Is there a reason why the RTD default cannot be consistent with default_branch from https://docs.github.com/en/rest/reference/repos ?

@stsewd
Copy link
Member

stsewd commented Mar 23, 2021

Is there a reason why the RTD default cannot be consistent with default_branch from https://docs.github.com/en/rest/reference/repos ?

The value is harcoded. We could make it dynamic, but not all repos are connected to a remote repository.

@stsewd
Copy link
Member

stsewd commented Mar 23, 2021

@this-josh looks like your project was building with main at the beginning, but them for some reason all versions disappeared. That's why you only see --- as option. Not sure how that happened. I have set the default branch to main manually for now. Maybe did you change your project to be single version?

https://readthedocs.org/projects/felling/builds/13325683/

Let us know if you see this happening again.

Edit: looks like your project isn't creating the versions, this may be a bug, I'm looking into that.

@stsewd
Copy link
Member

stsewd commented Mar 23, 2021

@this-josh I found the problem, you have two versions with the same name: Felling-v0.0.2 and felling-v0.0.2, we are creating the versions in bulk so our sluggify code isn't called as we expect :/. I'll send a PR, you can remove one of those in the meantime, or don't click "save" in the admin of your project till next week the fix is live, your project already has the "main" branch as default.

@this-josh
Copy link

@stsewd Thanks for manually setting it to main. No, I haven't used the single version option as ideally I'd like multiple versions eventually.

Thank you for finding that! This is the first time I've tried to properly version a package and I've certainly had a few hiccups along the way. I have removed the tag Felling-v0.0.2 as this one wasn't required, and I won't interact with the RTD admin section until next week.

@pradyunsg
Copy link

pradyunsg commented Apr 27, 2021

FYI: This affected at least 2 of PyPA's projects as well, and it is a very subtle breakage that isn't detected until someone notices things being missing in the changelog or something.

The fix is relatively straightforward, once you know what's happening though. I've described what I had to do for pypa/packaging in pypa/packaging#415 (comment).

@astrojuanlu astrojuanlu changed the title RTD assumes master for a default branch in git RTD does not detect when the default branch is renamed (e.g. from master to main) Apr 28, 2021
@astrojuanlu
Copy link
Contributor

astrojuanlu commented Apr 28, 2021

Hi all - we acknowledge this is affecting lots of projects these days, and that RTD is not very smart about it. We have some internal changes underway that could simplify this transition or make it invisible for users, but it will still take us some time to get there.

In the meanwhile, if you rename your default branch from master to main (or something else), please follow these steps to get the builds working again:

  1. Go to https://readthedocs.org/dashboard/{your_project}/advanced/, and change the "Default branch" from ----- to main (or whatever your default branch is)
  2. This will trigger a rebuild of your "Default version" (stable, latest, or something else). If needed, launch a rebuild of the version you are interested in.

From here, everything should work again.

Please notice that:

  • For newly imported projects, RTD will pick the correct default branch, as @humitos said above.
  • It is unlikely that we implement solutions like polling GitHub to detect branch renames, since the number of projects we host is just too large.

I am renaming this issue to clarify what's actually happening, and pinning it so it's more visible.

We will update here if we improve the detection of branch renames for projects linked to GitHub repositories. Sorry all for the inconvenience and thanks for your patience!

@astrojuanlu astrojuanlu pinned this issue Apr 28, 2021
@ericholscher ericholscher changed the title RTD does not detect when the default branch is renamed (e.g. from master to main) Detect when the default branch is renamed (e.g. from master to main) Jun 8, 2021
@stale
Copy link

stale bot commented Jul 24, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the Status: stale Issue will be considered inactive soon label Jul 24, 2021
@astrojuanlu astrojuanlu added Accepted Accepted issue on our roadmap and removed Status: stale Issue will be considered inactive soon labels Jul 24, 2021
humitos added a commit that referenced this issue Mar 2, 2022
Currently, we are using `Project.default_branch` when it's defined and falling
back to VCS's default branch. However, with the massive rename from `master` to
`main` in GitHub, a lot of project are pointing to the wrong default branch.

Since we are storing the default branch of the GitHub repository under
`RemoteRepository.default_branch` and we are refreshing this list more
frequently now, we can use it before using VCS's default branch as fallback.

Closes #7907
@humitos humitos unpinned this issue Mar 2, 2022
@simontorres
Copy link

I'm experiencing a similar issue now but after checking the option to build pull requests, the default branch is set to master which never existed in the first place and also I can't get the available branches in the dropdown menu to select the default branch.

@add-n2x
Copy link

add-n2x commented May 18, 2022

Same Issue. Just moved from master to main but RTD seems not updating the available branches.

Go to https://readthedocs.org/dashboard/{your_project}/advanced/, and change the "Default branch" from ----- to main (or whatever your default branch is)

@astrojuanlu this doesn't work. RTD doesn't display the main branch. Not even the new develop branch. I looks like the updating of branches is broken.

Is it possible to change it some other way? Here's the project: https://readthedocs.org/api/v3/projects/aura-automated-radio/

@add-n2x
Copy link

add-n2x commented May 18, 2022

I was able to update the default branch through API:

  1. Go to your RTD userprofile and generate an API Token.
  2. Run this in your command-line:
curl \
  -X PATCH \
  -H "Authorization: Token $TOKEN" https://readthedocs.org/api/v3/projects/$PROJECT/ \
  -H "Content-Type: application/json" \
  -d '{ "default_branch": "main" }'

Note, to replace your $TOKEN and $PROJECT.

But then, when checking for the default branch in the UI:

Go to https://readthedocs.org/dashboard/{your_project}/advanced/, and change the "Default branch" from ----- to main (or whatever your default branch is)

...this is now updated from "master" to "--------". When clicking the dropdown, there is still no "main" branch available.

But when triggering a manual build, now it is cloning from the main branch. After this build is completed, the main and other available branches are also updated in the versions view.

humitos added a commit that referenced this issue May 18, 2022
Currently, we are using `Project.default_branch` when it's defined and falling
back to VCS's default branch. However, with the massive rename from `master` to
`main` in GitHub, a lot of project are pointing to the wrong default branch.

Since we are storing the default branch of the GitHub repository under
`RemoteRepository.default_branch` and we are refreshing this list more
frequently now, we can use it before using VCS's default branch as fallback.

Closes #7907
humitos added a commit that referenced this issue May 18, 2022
Currently, we are using `Project.default_branch` when it's defined and falling
back to VCS's default branch. However, with the massive rename from `master` to
`main` in GitHub, a lot of project are pointing to the wrong default branch.

Since we are storing the default branch of the GitHub repository under
`RemoteRepository.default_branch` and we are refreshing this list more
frequently now, we can use it before using VCS's default branch as fallback.

Closes #7907
humitos added a commit that referenced this issue May 18, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Currently, we are using `Project.default_branch` when it's defined and falling
back to VCS's default branch. However, with the massive rename from `master` to
`main` in GitHub, a lot of project are pointing to the wrong default branch.

Since we are storing the default branch of the GitHub repository under
`RemoteRepository.default_branch` and we are refreshing this list more
frequently now, we can use it before using VCS's default branch as fallback.

Closes #7907
@humitos
Copy link
Member

humitos commented May 18, 2022

Once #8988 got merged and released, Read the Docs will autodetect the default branch defined by the VCS repository connected if the project was imported using the "Import Project" wizard (if it was manually imported it may not be connected to the VCS repository). However, it may require forcing a re-sync of the VCS data first:

  1. going to the "Import Project" page
  2. clicking on the double arrow at the top of the page to re-sync the VCS data

After that, if you have changed the default branch Read the Docs should use the new default branch set at your repository.

@georgedimitriadis
Copy link

I have the same problem as described above. My github repo never had a master branch and I am getting only an empty (------) option in the Default branch of the advanced settings.
My issue is that I am trying to import a github repo that belongs to an organisation I am the owner of (all public) and not directly to my account. The Import Project page does not show any of the repos in the organisation so I only can import manually.

@stsewd
Copy link
Member

stsewd commented May 31, 2022

@georgedimitriadis make sure that you have granted access to the Read the Docs OAuth App for your GitHub organization

@georgedimitriadis
Copy link

Thanks for the reply.

That was the solution to my problem.

@kevingreenman
Copy link

kevingreenman commented Apr 23, 2024

I'm having an issue with this while using RTD with the chemprop package. Chemprop has been on RTD for several years and has previously always had master as its primary branch. In the RTD settings, the default branch was never set (it said --------- and thus pointed to the default of master).

We recently released a new version of chemprop that we had been developing on our v2/dev branch, which was also active as a version on RTD. When we were ready to release our new stable version, we renamed v2/dev to main and then changed our primary branch in our GitHub repo settings from master to main. On RTD, I deactivated the v2/dev version and changed the default branch from --------- to main. The latest build now fails.

The error we're seeing is

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/sphinx/cmd/build.py", line 332, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/sphinx/application.py", line 268, in __init__
    self._init_builder()
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/sphinx/application.py", line 339, in _init_builder
    self.events.emit('builder-inited')
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/sphinx/events.py", line 97, in emit
    results.append(listener.handler(self.app, *args))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/autoapi/extension.py", line 124, in run_autoapi
    if sphinx_mapper_obj.load(
       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/autoapi/mappers/python/mapper.py", line 287, in load
    raise ExtensionError(f"No source files found in: {','.join(dirs)}")
sphinx.errors.ExtensionError: No source files found in: /home/docs/checkouts/readthedocs.org/user_builds/chemprop/checkouts/latest
Extension error:
No source files found in: /home/docs/checkouts/readthedocs.org/user_builds/chemprop/checkouts/latest

I created a new RTD version that points directly to main, and that build succeeds, even though the code is exactly the same as for the latest version.

We're also tracking this in our own issue: chemprop/chemprop#828

EDIT: this was resolved in #11321

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Accepted Accepted issue on our roadmap Support Support question
Projects
None yet
Development

Successfully merging a pull request may close this issue.