ICU-22986 GL takes CM

[eggrobin]

A bug in the production implementation, which was not caught in ICU 76 because it is duplicated in the new monkeys (whose rules are similar to the production rules in the ways that matter here) and because of an equivalent bug in the old old monkeys: we should have looked through CM* for GL in

icu/icu4c/source/test/intltest/rbbitst.cpp

Lines 3254 to 3266 in e000c5c

	// prevPosX2 is -1 if there was a break; but the UAX #14 rules can
	// look through breaks.
	int breakObliviousPrevPosX2 = fText->moveIndex32(prevPos, -1);
	if (fBK->contains(fText->char32At(breakObliviousPrevPosX2)) ||
	fCR->contains(fText->char32At(breakObliviousPrevPosX2)) ||
	fLF->contains(fText->char32At(breakObliviousPrevPosX2)) ||
	fNL->contains(fText->char32At(breakObliviousPrevPosX2)) ||
	fSP->contains(fText->char32At(breakObliviousPrevPosX2)) ||
	fGL->contains(fText->char32At(breakObliviousPrevPosX2)) ||
	fZW->contains(fText->char32At(breakObliviousPrevPosX2))) {
	setAppliedRule(pos, "LB 20a");
	continue;
	}

The new old monkeys (after ICU-22984=#3287) are much more principled about remap rules, actually modifying the working buffer, and so caught that.

Also fix an issue with insufficiently greedy regular expressions in the rules (new rules generated from unicode-org/unicodetools#988) and with remap rules creating characters out of surrogates (as in <lead, combining mark, trail>).

Checklist

• Required: Issue filed: ICU-22986
• Required: The PR title must be prefixed with a JIRA Issue number. Example: "ICU-1234 Fix xyz"
• Required: The PR description must include the link to the Jira Issue, for example by completing the URL in the first checklist item
• Required: Each commit message must be prefixed with a JIRA Issue number. Example: "ICU-1234 Fix xyz"
• Issue accepted (done by Technical Committee after discussion)
• Tests included, if applicable
• API docs and/or User Guide docs changed or added, if applicable

[jira-pull-request-webhook]

Hooray! The files in the branch are the same across the force-push. 😃

~ Your Friendly Jira-GitHub PR Checker Bot

[jira-pull-request-webhook]

Notice: the branch changed across the force-push!

• icu4c/source/test/testdata/rbbitst.txt is now changed in the branch
• icu4j/main/core/src/test/resources/com/ibm/icu/dev/test/rbbi/rbbitst.txt is now changed in the branch

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

[markusicu]

changes lgtm (=as advertised)
i need to figure out a new recipe for flipping the bytes, since the tool is shifting

[aheninger]

The changes look good, but they also need to be propagated into the various line rule tailorings, line_cj.txt, etc.

[eggrobin]

Thanks for pointing that out @aheninger. Done.

[FrankYFTang]

The Exhaustive Tests for ICU #22 was broken between e025466 and 2e57f07 in TestMonkey

https://github.com/unicode-org/icu/actions/runs/12209763924

Likely caused by this #3287 . Is this intend to fix that?

[eggrobin]

The Exhaustive Tests for ICU […] was broken […] by […] #3287. Is this intend[ed] to fix that?

Yes. See ICU-22986 and the discussion there.

[jira-pull-request-webhook]

Hooray! The files in the branch are the same across the force-push. 😃

~ Your Friendly Jira-GitHub PR Checker Bot