Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jj should be able to handle conflicts in executable scripts #1279

Closed
isinyaaa opened this issue Feb 17, 2023 · 13 comments
Closed

jj should be able to handle conflicts in executable scripts #1279

isinyaaa opened this issue Feb 17, 2023 · 13 comments
Assignees

Comments

@isinyaaa
Copy link
Contributor

Description

Currently, if jj notices that a file that results in a conflict is an executable it won't bother helping the user resolve it, even if it's a script.

Steps to Reproduce the Problem

  1. Generate a conflict with an executable script (.sh for example, marked with chmod +x)
  2. Run jj resolve

Expected Behavior

As it's quite usual to edit such scripts, jj should allow for conflict resolution.

Actual Behavior

jj errors out of the conflict resolution.

@martinvonz
Copy link
Member

martinvonz commented Feb 17, 2023

Definitely. I happened to be reminded of that while looking at that code just an hour or two ago. We can probably have this function here return whether the merged file should be executable: https://github.com/martinvonz/jj/blob/3dfedf581458f88982289f58db884bc65f1e635b/lib/src/conflicts.rs#L130-L144

Interested in attempting that?

@martinvonz
Copy link
Member

Oh, in the interactive case (which is what you actually asked about), the code is here:
https://github.com/martinvonz/jj/blob/3dfedf581458f88982289f58db884bc65f1e635b/src/merge_tools.rs#L171-L183

So we'd probably want extract_file_conflict_as_single_hunk() to return a (ConflictHunk, bool) pair, or maybe a custom type for that.

@isinyaaa
Copy link
Contributor Author

Interested in attempting that?

Maybe... I've studied some rust in the past so it's not something I'm very familiar with. But I do want to contribute :)

@ilyagr
Copy link
Contributor

ilyagr commented Feb 18, 2023

You'll also need to consider the case when one of the sides is executable and the other isn't. At first, I would suggest just aborting in that case, just like jj does now for all executables.

@martinvonz
Copy link
Member

You'll also need to consider the case when one of the sides is executable and the other isn't. At first, I would suggest just aborting in that case, just like jj does now for all executables.

Yes, that's fine to do at first. It should be easy to add support for merging the executable bit in the common case of a regular 3-way merge, however.

@ilyagr
Copy link
Contributor

ilyagr commented Mar 3, 2023

You'll also need to consider the case when one of the sides is executable and the other isn't. At first, I would suggest just aborting in that case, just like jj does now for all executables.

We discussed this on Discord, and it turns out that this situation is simpler than I thought. At least for 3-way merges, you know whether the base had the executable bit set, so it's a conflict between one side that didn't change it and another side that did change it. The latter can simply win.

@ilyagr
Copy link
Contributor

ilyagr commented Jun 6, 2023

A related feature would be a jj chmod command so that you can do jj chmod +x file, even on Windows or if file is conflicted.

Without such a command, tests for conflict resolution of executable bits would be harder to write and would have to be specific to POSIX systems.

Update: This was done in #1693

@qfel
Copy link
Contributor

qfel commented Aug 8, 2023

I was looking at it a bit, but I'm not particularly confident it's as simple as the comments suggest.

IIUC there's a few parts:

  • How a conflict is signaled. Currently it seems it's purely by writing to a file. If both sides are regular files, diff markers are inserted. In any other case (in particular, if at least one is executable), the file contents just contain a text representation of the conflict struct (no diffs, just file IDs).
  • How a conflict is resolved. Intuitively I'd expect by getting rid of diff markers, or explicitly setting the mode on the file (but that requires a dedicated jj chmod, even on POSIX). But right now it seems any kind of modification to the file will resolve the commit?
  • There also seems to be some code that parses back the conflict markers, but I'm not sure what it's used for.
  • And finally, how a conflict is generated in the internal structures. In particular, updating the code that formats conflicts to simply show the diff with no regard to the executable bit may work if the conflict-detection logic is capable of merging executable bits, but if there are cases when the executable bit itself is in a conflicted state (eg. both sides added the same file with different exec bit) - then this will be a lossy way to signal to the user what is conflicted.

What is the end goal here? Do we want the user to look at the conflicted file, and be able to determine what the conflict is about? Or do we expect they may need to jj status or something similar to learn more details (eg. file vs directory, or regular vs exec file).

@ilyagr
Copy link
Contributor

ilyagr commented Aug 8, 2023

jj chmod was implemented in #1693. 😀 It doesn't solve this issue completely, but should be a good workaround. Try it out, let me know how it works for you.

@martinvonz
Copy link
Member

I was looking at it a bit, but I'm not particularly confident it's as simple as the comments suggest.

IIUC there's a few parts:

Thank you :) Good analysis. I'll answer them out of order.

What is the end goal here? Do we want the user to look at the conflicted file, and be able to determine what the conflict is about? Or do we expect they may need to jj status or something similar to learn more details (eg. file vs directory, or regular vs exec file).

I don't think we should embed that information in the file contents. And yes, I think jj status and jj resolve list should list the conflict and somehow indicate conflict in the executable bit. Since it's a single bit, it's probably enough to just say that there is a conflict in it - I don't think we need to say which file mode is associated with which file contents.

  • How a conflict is signaled. Currently it seems it's purely by writing to a file. If both sides are regular files, diff markers are inserted. In any other case (in particular, if at least one is executable), the file contents just contain a text representation of the conflict struct (no diffs, just file IDs).

Regardless of the executable bit, we should write the conflict markers to the file. If there was a conflict in the executable bit, I guess it makes most sense to make the file not executable. Otherwise (no conflict in the executable bit), we should use that file mode.

  • How a conflict is resolved. Intuitively I'd expect by getting rid of diff markers, or explicitly setting the mode on the file (but that requires a dedicated jj chmod, even on POSIX). But right now it seems any kind of modification to the file will resolve the commit?

When the conflicts in the file have been resolved, we need to figure out what to do with the executable bit. If the file is now executable on disk, it means that we had made it executable because there was no conflict, or the user had made it executable later. Either way, I suppose we should record the conflict as resolved with the executable bit set . Otherwise we need to look at the conflict from the tree again. If the executable bit can be automatically resolved, we should use that value on Windows or the value on disk on Unix. Otherwise (conflict in exec bit), we'll have to leave the whole conflict unresolved. And as Ilya said, jj chmod should allow the user to resolve conflicts in the executable bit.

  • There also seems to be some code that parses back the conflict markers, but I'm not sure what it's used for.

That's so you can partially resolve a conflict and continue to work on the resolution later. I don't think you need to worry about it in the context of this bug.

  • And finally, how a conflict is generated in the internal structures. In particular, updating the code that formats conflicts to simply show the diff with no regard to the executable bit may work if the conflict-detection logic is capable of merging executable bits, but if there are cases when the executable bit itself is in a conflicted state (eg. both sides added the same file with different exec bit) - then this will be a lossy way to signal to the user what is conflicted.

I suspect it's fine to ignore the executable bit in most of those cases, such as when displaying a diff.

@martinvonz
Copy link
Member

@qfel, I've been refactoring the working-copy code a lot lately (most recently #2046). I don't know if you've started working on what we talked about earlier, but note that my changes will conflict with that work. I hope it'll make it easier, however.

@qfel
Copy link
Contributor

qfel commented Aug 13, 2023

Changing the materialize code seemed simple enough, but I couldn't shake the feeling I'm missing something so was sporadically digging a bit into code instead of sending out PRs. I guess I'll take a look again once #2406 is in and maybe send something simple and let the reviewers worry instead.

@martinvonz
Copy link
Member

Changing the materialize code seemed simple enough, but I couldn't shake the feeling I'm missing something so was sporadically digging a bit into code instead of sending out PRs. I guess I'll take a look again once #2406 is in and maybe send something simple and let the reviewers worry instead.

Sounds good. We're happy to help find any missing pieces during review.

@martinvonz martinvonz self-assigned this Oct 24, 2023
martinvonz added a commit that referenced this issue Oct 24, 2023
AFAICT, all callers of `Merge::to_file_merge()` are already well
prepared for working with executable files. It's called from these
places:

* `local_working_copy.rs`: Materialized conflicts are correctly
  updated using `Merge::with_new_file_ids()`.

* `merge_tools/`: Same as above.

* `cmd_cat()`: We already ignore the executable bit when we print
  non-conflicted files, so it makes sense to also ignore it for
  conflicted files.

* `git_diff_part()`: We print all conflicts with mode "100644" (the
  mode for regular files). Maybe it's best to use "100755" for
  conflicts that are unambiguously executable, or maybe it's better to
  use a fake mode like "000000" for all conflicts. Either way, the
  current behavior seems fine.

* `diff_content()`: We use the diff content in various diff
  formats. We could add more detail about the executable bits in some
  of them, but I think the current output is fine. For example,
  instead of our current "Created conflict in my-file", we could say
  "Created conflict in executable file my-file" or "Created conflict
  in ambiguously executable file my-file". That's getting verbose,
  though.

So, I think all we need to do is to make `Merge::to_file_merge()` not
require its inputs to be non-executable.

Closes #1279.
martinvonz added a commit that referenced this issue Oct 24, 2023
AFAICT, all callers of `Merge::to_file_merge()` are already well
prepared for working with executable files. It's called from these
places:

* `local_working_copy.rs`: Materialized conflicts are correctly
  updated using `Merge::with_new_file_ids()`.

* `merge_tools/`: Same as above.

* `cmd_cat()`: We already ignore the executable bit when we print
  non-conflicted files, so it makes sense to also ignore it for
  conflicted files.

* `git_diff_part()`: We print all conflicts with mode "100644" (the
  mode for regular files). Maybe it's best to use "100755" for
  conflicts that are unambiguously executable, or maybe it's better to
  use a fake mode like "000000" for all conflicts. Either way, the
  current behavior seems fine.

* `diff_content()`: We use the diff content in various diff
  formats. We could add more detail about the executable bits in some
  of them, but I think the current output is fine. For example,
  instead of our current "Created conflict in my-file", we could say
  "Created conflict in executable file my-file" or "Created conflict
  in ambiguously executable file my-file". That's getting verbose,
  though.

So, I think all we need to do is to make `Merge::to_file_merge()` not
require its inputs to be non-executable.

Closes #1279.
martinvonz added a commit that referenced this issue Oct 24, 2023
AFAICT, all callers of `Merge::to_file_merge()` are already well
prepared for working with executable files. It's called from these
places:

* `local_working_copy.rs`: Materialized conflicts are correctly
  updated using `Merge::with_new_file_ids()`.

* `merge_tools/`: Same as above.

* `cmd_cat()`: We already ignore the executable bit when we print
  non-conflicted files, so it makes sense to also ignore it for
  conflicted files.

* `git_diff_part()`: We print all conflicts with mode "100644" (the
  mode for regular files). Maybe it's best to use "100755" for
  conflicts that are unambiguously executable, or maybe it's better to
  use a fake mode like "000000" for all conflicts. Either way, the
  current behavior seems fine.

* `diff_content()`: We use the diff content in various diff
  formats. We could add more detail about the executable bits in some
  of them, but I think the current output is fine. For example,
  instead of our current "Created conflict in my-file", we could say
  "Created conflict in executable file my-file" or "Created conflict
  in ambiguously executable file my-file". That's getting verbose,
  though.

So, I think all we need to do is to make `Merge::to_file_merge()` not
require its inputs to be non-executable.

Closes #1279.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants