Format type hints after comments

[charlie572]

Description

This fixes #3350.

The problem is that when Black encounters standalone comments on one line inside brackets, it splits the line using delimiter_split. This will split on the comment, but it will also split the line on the highest priority delimiter, which isn't always necessary.

I've added a comma_split function, which is the same as delimiter_split, but delimiter_priority = COMMA_PRIORITY. Black will now try comment_split before it tries delimiter_split. This means it will first split on standalone comments and commas, and only split on other delimiters if the lines are still too long after that.

Checklist - did you ...

• Add an entry in CHANGES.md if necessary?
• Add / update tests if necessary?
• Add new / update outdated documentation?

[github-actions]

diff-shades results comparing this PR (029fce9) to main (3a2d76c). The full diff is available in the logs under the "Generate HTML diff report" step.

╭─────────────────────── Summary ────────────────────────╮
│ 6 projects & 10 files changed / 44 changes [+11/-33]   │
│                                                        │
│ ... out of 2 495 262 lines, 11 738 files & 23 projects │
╰────────────────────────────────────────────────────────╯

Differences found.

What is this? | Workflow run | diff-shades documentation

[charlie572]

Sorry, I should have run diff-shades before submitting a pull request. I'll keep working on this, and see if I can fix this solution or think of a better one.

[JelleZijlstra]

It's fine to rely on CI to run diff-shades for you. I haven't looked closely here but probably you just need to gate your changes to the preview style.

[charlie572]

I think I've got a better solution now.

If there are comments at the beginning or end of the line, I call standalone_comment_split before delimiter_split. Else, I call delimiter_split first. I'm not using the comma_split function I wrote any more.

This means that now, when there is a comment in the middle of an expression or list, it will perform a delimiter split.

(
	a
	and b
	# comment
	and c
	and d
)

But when the comment is at the top or bottom, it won't perform a delimiter split (if it doesn't have to).

(
	# comment
	a and b and c and d
)

Currently, Black performs a delimiter split in both cases, which is unnecessary.

[charlie572]

Should I rebase main onto here to resolve the merge conflict, or will the maintainer do that? The only merge conflict is one line in CHANGES.md.
Also, why isn't CI running?

[JelleZijlstra]

Easiest to fix simple merge conflicts by going through GitHub's UI, I just did it for you.

[charlie572]

Okay, thanks.

[aqeelat]

@charlie572 I'm interested in this PR. Can you please rebase?

[charlie572]

Great. I can do this tomorrow. Do I just need to rebase main onto this branch to fix merge conflicts?

[hauntsaninja]

Looking at the diff shades report https://github.com/psf/black/actions/runs/5431147643/jobs/9880366015?pr=3362 it looks this causes the "magic" trailing comma to not get respected inside collection literals if there's a comment

[charlie572]

I'm confused about the magic_trailing_comma attribute of the Line class. Why does it point at the closing bracket after the magic trailing comma, and not at the magic trailing comma token? This means that my code can't check for the magic trailing comma, since it is looking at a Line object containing just the comma-separated members, not the surrounding brackets.

I can think of two ways to make it handle magic trailing commas properly:
a. add another attribute that points at the magic trailing comma token
b. search the tokens for a magic trailing comma inside the transform_line function (around line 600).

Option a makes the most sense to me. Is that okay?

[charlie572]

I implemented option a. But now the attribute names are weird. Maybe magic_trailing_comma should point at the comma, and bracket_after_magic_trailing_comma can point at the following bracket? Or maybe we can remove the latter attribute?

[aqeelat]

@charlie572 I understand nothing from the code in this PR (I'm not familiar with the internals of Black) but I want to thank you for your effort.

[charlie572]

Thank you.