-
-
Notifications
You must be signed in to change notification settings - Fork 919
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Override bash executable, defaulting to Git for Windows git bash over WSL bash #1791
base: main
Are you sure you want to change the base?
Changes from all commits
77f74ce
4bde10d
6676668
50c74f4
bf919f6
f065d1f
0c14cac
8200ad1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -35,11 +35,11 @@ jobs: | |
python-version: ${{ matrix.python-version }} | ||
allow-prereleases: ${{ matrix.experimental }} | ||
|
||
- name: Set up WSL (Windows) | ||
if: startsWith(matrix.os, 'windows') | ||
uses: Vampire/[email protected] | ||
with: | ||
distribution: Debian | ||
# - name: Set up WSL (Windows) | ||
# if: startsWith(matrix.os, 'windows') | ||
# uses: Vampire/[email protected] | ||
# with: | ||
# distribution: Debian | ||
|
||
- name: Prepare this repo for tests | ||
run: | | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,6 +15,7 @@ | |
import subprocess | ||
import threading | ||
from textwrap import dedent | ||
from pathlib import Path | ||
|
||
from git.compat import defenc, force_bytes, safe_decode | ||
from git.exc import ( | ||
|
@@ -360,9 +361,107 @@ def __setstate__(self, d: Dict[str, Any]) -> None: | |
the top level ``__init__``. | ||
""" | ||
|
||
_bash_exec_env_var = "GIT_PYTHON_BASH_EXECUTABLE" | ||
|
||
bash_exec_name = "bash" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems not to be used. I'm not sure it's needed. If it is kept, then |
||
"""Default bash command.""" | ||
|
||
GIT_PYTHON_BASH_EXECUTABLE = None | ||
""" | ||
Provides the path to the bash executable used for commit hooks. This is | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I recommend adding "on Windows" to the first sentence, both here and in the |
||
ordinarily set by `Git.refresh_bash()`. Note that the default behavior of | ||
invoking commit hooks on Windows has changed to not prefer WSL bash with | ||
the introduction of this variable. See the `Git.refresh_bash()` | ||
documentation for details on the default values and search paths. | ||
""" | ||
|
||
@classmethod | ||
def _get_default_bash_path(cls) -> str: | ||
# Assumes that, if user is running in Windows, they probably are using | ||
# Git for Windows, which includes Git BASH and should be associated | ||
# with the configured Git command set in `refresh()`. | ||
# Uses the output of `git --exec-path` for the currently configured | ||
# Git command to find its `git-core` directory. If one assumes that | ||
# the `git-core` directory is always three levels deeper than the | ||
# root directory of the Git installation, we can try going up three | ||
# levels and then navigating to (root)/bin/bash.exe. If this exists, | ||
# prefer it over the WSL version in System32, direct access to which | ||
# is reportedly deprecated. Fail back to default "bash.exe" if | ||
# the Git for Windows lookup doesn't work. | ||
emanspeaks marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# | ||
# This addresses issues where git hooks are intended to run assuming | ||
# the "native" Windows environment as seen by git.exe rather than | ||
# inside the git sandbox of WSL, which is likely configured | ||
# independently of the Windows Git. A noteworthy example are repos | ||
# with Git LFS, where Git LFS may be installed in Windows but not | ||
# in WSL. | ||
emanspeaks marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if os.name != "nt": | ||
return "bash" | ||
gitcore = Path(cls()._call_process("--exec-path")) | ||
gitroot = gitcore.parent.parent.parent | ||
gitbash = gitroot / "bin" / "bash.exe" | ||
return str(gitbash) if gitbash.exists() else "bash.exe" | ||
|
||
@classmethod | ||
def refresh_bash(cls, path: Union[None, PathLike] = None) -> bool: | ||
emanspeaks marked this conversation as resolved.
Show resolved
Hide resolved
|
||
""" | ||
Refreshes the cached path to the bash executable used for executing | ||
commit hook scripts. This gets called by the top-level `refresh()` | ||
Comment on lines
+408
to
+409
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As above for |
||
function on initial package import (see the top level __init__), but | ||
this method may be invoked manually if the path changes after import. | ||
|
||
This method only checks one path for a valid bash executable at a time, | ||
using the first non-empty path provided in the following priority | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The This description could be changed to just say "the first path provided..." because it is reasonable to interpret an empty environment variable as not providing a path, which is what I would suggest. But if you feel that's not accurate enough, then it could be expanded to be more specific, or something about the environment variable's value being nonempty could be added in item 2 of the numbered list. |
||
order: | ||
|
||
1. the explicit `path` argument to this method | ||
2. the environment variable `GIT_PYTHON_BASH_EXECUTABLE` if it is set | ||
and available via `os.environ` upon calling this method | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this line about It's possible to set environment variables in a way that does not make them available in |
||
3. if the current platform is not Windows, the simple string `"bash"` | ||
4. if the current platform is Windows, inferred from the current | ||
provided Git executable assuming it is part of a Git for Windows | ||
distribution. | ||
Comment on lines
+420
to
+423
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This omits the final fallback of |
||
|
||
The current platform is checked based on the call `os.name`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this is either an implementation detail, or an attempt to state something about the behavior that should be stated directly instead, and that it could be removed. But this is another subjective thing that I think you should feel free to leave as is, if you believe it is best to have it. If any information related to the platform check is given, then I think the most relevant is that Cygwin isn't treated as Windows for this purpose. It is not treated as Windows for most purposes (in GitPython and also more broadly) so I don't think that has to be said, but the confusion that led to |
||
|
||
This is a change to the default behavior from previous versions of | ||
GitPython. In the event backwards compatibility is needed, the `path` | ||
argument or the environment variable may be set to the string | ||
`"bash.exe"`, which on most systems invokes the WSL bash by default. | ||
|
||
This change to default behavior addresses issues where git hooks are | ||
intended to run assuming the "native" Windows environment as seen by | ||
git.exe rather than inside the git sandbox of WSL, which is likely | ||
configured independently of the Windows Git. A noteworthy example are | ||
repos with Git LFS, where Git LFS may be installed in Windows but not | ||
in WSL. | ||
""" | ||
# Discern which path to refresh with. | ||
if path is not None: | ||
new_bash = os.path.expanduser(path) | ||
# new_bash = os.path.abspath(new_bash) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This commented-out line can be removed. If it is valuable to note that this step is not being done, then a comment could be added stating the relevant behavior. |
||
else: | ||
new_bash = os.environ.get(cls._bash_exec_env_var) | ||
if not new_bash: | ||
new_bash = cls._get_default_bash_path() | ||
|
||
# Keep track of the old and new bash executable path. | ||
# old_bash = cls.GIT_PYTHON_BASH_EXECUTABLE | ||
Comment on lines
+448
to
+449
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This commented-out code should be removed, including the comment above it that only applies to the now-commented-out line. |
||
cls.GIT_PYTHON_BASH_EXECUTABLE = new_bash | ||
|
||
# Test if the new git executable path exists. | ||
has_bash = Path(cls.GIT_PYTHON_BASH_EXECUTABLE).exists() | ||
return has_bash | ||
|
||
@classmethod | ||
def refresh(cls, path: Union[None, PathLike] = None) -> bool: | ||
"""This gets called by the refresh function (see the top level __init__).""" | ||
""" | ||
This gets called by the refresh function (see the top level __init__). | ||
|
||
Note that calling this method directly does not automatically update | ||
the cached path to `bash`; either invoke the top level `refresh()` | ||
function or call `Git.refresh_bash()` directly. | ||
""" | ||
# Discern which path to refresh with. | ||
if path is not None: | ||
new_git = os.path.expanduser(path) | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1019,16 +1019,16 @@ class Mocked: | |
rel = index._to_relative_path(path) | ||
self.assertEqual(rel, os.path.relpath(path, root)) | ||
|
||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.Absent, | ||
reason="Can't run a hook on Windows without bash.exe.", | ||
rasies=HookExecutionError, | ||
) | ||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
raises=HookExecutionError, | ||
) | ||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.Absent, | ||
# reason="Can't run a hook on Windows without bash.exe.", | ||
# rasies=HookExecutionError, | ||
# ) | ||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
# reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
# raises=HookExecutionError, | ||
# ) | ||
Comment on lines
+1022
to
+1031
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These commented-out decorators can be removed. I don't think we'll need them anymore, and they'll remain in the history, so they can be brought back if needed. (As in For tests, including this one, that are neither regression tests for security vulnerabilities nor tests of what happens when running hooks fails, I don't think there's any need to cover WSL or absent If |
||
@with_rw_repo("HEAD", bare=True) | ||
def test_run_commit_hook(self, rw_repo): | ||
index = rw_repo.index | ||
|
@@ -1064,41 +1064,41 @@ def test_hook_uses_shell_not_from_cwd(self, rw_dir, case): | |
shutil.copy(fixture_path("polyglot"), hook_path("polyglot", repo.git_dir)) | ||
payload = Path(rw_dir, "payload.txt") | ||
|
||
if type(_win_bash_status) in {WinBashStatus.Absent, WinBashStatus.WslNoDistro}: | ||
# The real shell can't run, but the impostor should still not be used. | ||
with self.assertRaises(HookExecutionError): | ||
with maybe_chdir: | ||
run_commit_hook("polyglot", repo.index) | ||
self.assertFalse(payload.exists()) | ||
else: | ||
# The real shell should run, and not the impostor. | ||
with maybe_chdir: | ||
run_commit_hook("polyglot", repo.index) | ||
self.assertFalse(payload.exists()) | ||
output = Path(rw_dir, "output.txt").read_text(encoding="utf-8") | ||
self.assertEqual(output, "Ran intended hook.\n") | ||
|
||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.Absent, | ||
reason="Can't run a hook on Windows without bash.exe.", | ||
rasies=HookExecutionError, | ||
) | ||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
raises=HookExecutionError, | ||
) | ||
# if type(_win_bash_status) in {WinBashStatus.Absent, WinBashStatus.WslNoDistro}: | ||
# # The real shell can't run, but the impostor should still not be used. | ||
# with self.assertRaises(HookExecutionError): | ||
# with maybe_chdir: | ||
# run_commit_hook("polyglot", repo.index) | ||
# self.assertFalse(payload.exists()) | ||
# else: | ||
Comment on lines
+1067
to
+1073
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure what the best thing is to do here right now. Some of the modifications to the test suite to take advantage of the improvements in this pull request could be done later. The purpose of this test method is to verify that an untrusted repository (or, more realistically, an untrusted branch, possibly from a fork) with a malicious It is a shortcoming of #1792, which fixed CVE-2024-22190 and added this test, that both scenarios would not be tested in a single run of the test suite. This would have been tricky to do prior to the changes in this pull request, but I probably could and maybe should have done it. Instead, this tested whichever of the two scenarios applied to the test system. Both scenarios were common prior to the changes in this pull request, which makes the broken-or-unavailable Possible approaches:
I think any of those approaches, and probably various others I have not thought of, are reasonable. |
||
# The real shell should run, and not the impostor. | ||
with maybe_chdir: | ||
run_commit_hook("polyglot", repo.index) | ||
self.assertFalse(payload.exists()) | ||
output = Path(rw_dir, "output.txt").read_text(encoding="utf-8") | ||
self.assertEqual(output, "Ran intended hook.\n") | ||
|
||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.Absent, | ||
# reason="Can't run a hook on Windows without bash.exe.", | ||
# rasies=HookExecutionError, | ||
# ) | ||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
# reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
# raises=HookExecutionError, | ||
# ) | ||
Comment on lines
+1081
to
+1090
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As on |
||
@with_rw_repo("HEAD", bare=True) | ||
def test_pre_commit_hook_success(self, rw_repo): | ||
index = rw_repo.index | ||
_make_hook(index.repo.git_dir, "pre-commit", "exit 0") | ||
index.commit("This should not fail") | ||
|
||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
raises=AssertionError, | ||
) | ||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
# reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
# raises=AssertionError, | ||
# ) | ||
Comment on lines
+1097
to
+1101
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As on The situation with the This is roughly analogous to the situation in The possible approaches to |
||
@with_rw_repo("HEAD", bare=True) | ||
def test_pre_commit_hook_fail(self, rw_repo): | ||
index = rw_repo.index | ||
|
@@ -1121,21 +1121,21 @@ def test_pre_commit_hook_fail(self, rw_repo): | |
else: | ||
raise AssertionError("Should have caught a HookExecutionError") | ||
|
||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.Absent, | ||
reason="Can't run a hook on Windows without bash.exe.", | ||
rasies=HookExecutionError, | ||
) | ||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.Wsl, | ||
reason="Specifically seems to fail on WSL bash (in spite of #1399)", | ||
raises=AssertionError, | ||
) | ||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
raises=HookExecutionError, | ||
) | ||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.Absent, | ||
# reason="Can't run a hook on Windows without bash.exe.", | ||
# rasies=HookExecutionError, | ||
# ) | ||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.Wsl, | ||
# reason="Specifically seems to fail on WSL bash (in spite of #1399)", | ||
# raises=AssertionError, | ||
# ) | ||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
# reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
# raises=HookExecutionError, | ||
# ) | ||
Comment on lines
+1124
to
+1138
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As on I am not concerned about losing mention of the "Specifically seems to fail on WSL bash (in spite of #1399)" situation here, and I don't think even that commented-out decoration needs to be kept as comments here. After the improvements of this PR, I think most use of the WSL bash for hooks will be for people whose workflows were already working and who need it for backward compatibility. The solution to this problem, for people who experience it, is just to use the new default behavior. If the #1399-related failure with WSL is considered important to keep track of, then a new issue can be opened about it (I would be pleased to do this on request), or a comment could briefly mention it, but it seems to me that neither is necessary at this time. |
||
@with_rw_repo("HEAD", bare=True) | ||
def test_commit_msg_hook_success(self, rw_repo): | ||
commit_message = "commit default head by Frèderic Çaufl€" | ||
|
@@ -1149,11 +1149,11 @@ def test_commit_msg_hook_success(self, rw_repo): | |
new_commit = index.commit(commit_message) | ||
self.assertEqual(new_commit.message, "{} {}".format(commit_message, from_hook_message)) | ||
|
||
@pytest.mark.xfail( | ||
type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
raises=AssertionError, | ||
) | ||
# @pytest.mark.xfail( | ||
# type(_win_bash_status) is WinBashStatus.WslNoDistro, | ||
# reason="Currently uses the bash.exe of WSL, even with no WSL distro installed", | ||
# raises=AssertionError, | ||
# ) | ||
Comment on lines
+1152
to
+1156
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As on |
||
@with_rw_repo("HEAD", bare=True) | ||
def test_commit_msg_hook_fail(self, rw_repo): | ||
index = rw_repo.index | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the improvement from this PR, we shouldn't need to install a WSL distribution on CI anymore, so this commented-out code can be removed altogether. It will still be in the history and can be brought back if needed in the future. (I'm guessing you may have been planning to do this anyway, but I figured I'd mention it where applicable.)