Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add locale-sensitive casing #880

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Add locale-sensitive casing #880

wants to merge 6 commits into from

Conversation

mosuem
Copy link
Member

@mosuem mosuem commented Aug 28, 2024

Fixes #229
Fixes #336

  • Works for ECMA
  • Works for ICU4X - needs the right feature flags in the library build @robertbastian icu_casemap

Needs to land the artifact changes separately before landing this PR.


  • I’ve reviewed the contributor guide and applied the relevant portions to this PR.
Contribution guidelines:

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

Copy link

github-actions bot commented Aug 28, 2024

PR Health

Breaking changes ✔️
Package Change Current Version New Version Needed Version Looking good?
intl4x Non-Breaking 0.10.1 0.10.2-wip 0.10.2 ✔️
Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

Coverage ⚠️
File Coverage
pkgs/intl4x/hook/build.dart 💔 Not covered
pkgs/intl4x/lib/case_mapping.dart 💚 50 %
pkgs/intl4x/lib/intl4x.dart 💚 65 %
pkgs/intl4x/lib/src/case_mapping/case_mapping.dart 💚 83 %
pkgs/intl4x/lib/src/case_mapping/case_mapping_4x.dart 💚 50 %
pkgs/intl4x/lib/src/case_mapping/case_mapping_ecma.dart 💚 100 %
pkgs/intl4x/lib/src/case_mapping/case_mapping_impl.dart 💚 100 %
pkgs/intl4x/lib/src/case_mapping/case_mapping_stub.dart 💔 0 % ⬇️ NaN %
pkgs/intl4x/lib/src/case_mapping/case_mapping_stub_4x.dart 💔 0 % ⬇️ NaN %

This check for test coverage is informational (issues shown here will not fail the PR).

This check can be disabled by tagging the PR with skip-coverage-check.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbols
License Headers ✔️
// Copyright (c) 2024, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
Files
no missing headers

All source files should start with a license header.

Package publish validation ✔️
Package Version Status
package:intl 0.20.0-wip WIP (no publish necessary)
package:intl4x 0.10.2-wip WIP (no publish necessary)
package:intl_translation 0.20.1-wip WIP (no publish necessary)
package:messages 0.2.0 already published at pub.dev
package:messages_builder 0.2.1 already published at pub.dev
package:messages_serializer 0.2.1 already published at pub.dev
package:messages_shrinker 0.2.2-wip WIP (no publish necessary)

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

Copy link

@gnprice gnprice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent, looking forward to having this!

pkgs/intl4x/lib/case_mapping.dart Outdated Show resolved Hide resolved
@mosuem mosuem marked this pull request as ready for review September 6, 2024 11:28
@mosuem mosuem requested a review from robertbastian September 6, 2024 11:29
@github-actions github-actions bot added the type-infra A repository infrastructure change or enhancement label Sep 6, 2024
void main() {
testWithFormatting('test name', () {
const enUS = Locale(language: 'en', region: 'US');
expect('İstanbul'.toLocaleLowerCase(enUS), 'i̇stanbul');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's too many dots on the lowercase i. Does this pass?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the same result as I get from the corresponding browser API:

> "İstanbul".toLocaleLowerCase('en-US')
'i̇stanbul'
> "İstanbul".toLocaleLowerCase('en-US')[1]
'̇'

The result changes with a Turkish locale, to be what I think you're expecting:

> "İstanbul".toLocaleLowerCase('tr-TR')
'istanbul'
> "İstanbul".toLocaleLowerCase('tr-TR')[1]
's'

This test should probably exercise the Turkish locale too:

Suggested change
expect('İstanbul'.toLocaleLowerCase(enUS), 'i̇stanbul');
expect('İstanbul'.toLocaleLowerCase(enUS), 'i̇stanbul');
expect('İstanbul'.toLocaleLowerCase(trTR), 'istanbul');

so that the contrast is explicit. (It already does something very similar below on lines 16–17, but it's not obvious to the reader what that means for this "İstanbul" case.)

Copy link
Collaborator

@robertbastian robertbastian Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah that makes sense

void main() {
testWithFormatting('test name', () {
const enUS = Locale(language: 'en', region: 'US');
expect('İstanbul'.toLocaleLowerCase(enUS), 'i̇stanbul');
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the same result as I get from the corresponding browser API:

> "İstanbul".toLocaleLowerCase('en-US')
'i̇stanbul'
> "İstanbul".toLocaleLowerCase('en-US')[1]
'̇'

The result changes with a Turkish locale, to be what I think you're expecting:

> "İstanbul".toLocaleLowerCase('tr-TR')
'istanbul'
> "İstanbul".toLocaleLowerCase('tr-TR')[1]
's'

This test should probably exercise the Turkish locale too:

Suggested change
expect('İstanbul'.toLocaleLowerCase(enUS), 'i̇stanbul');
expect('İstanbul'.toLocaleLowerCase(enUS), 'i̇stanbul');
expect('İstanbul'.toLocaleLowerCase(trTR), 'istanbul');

so that the contrast is explicit. (It already does something very similar below on lines 16–17, but it's not obvious to the reader what that means for this "İstanbul" case.)

Comment on lines +19 to +20
final locales = ['tr', 'TR', 'tr-TR', 'tr-u-co-search', 'tr-x-turkish']
.map(Locale.parse);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ISTR there's one other case besides Turkish where locale-specific case-mapping is interesting — I think it was Lithuanian. It'd be nice to have a test case demonstrating that too.

Comment on lines +16 to +17
expect('\u0130'.toLocaleLowerCase(const Locale(language: 'tr')), 'i');
expect('\u0130'.toLocaleLowerCase(enUS), isNot('i'));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I like the Locale.parse way of expressing the locales that you use below — those strings like en-US are nice and compact, and are familiar from how locales are identified in other systems. Consider using it throughout the test:

Suggested change
expect('\u0130'.toLocaleLowerCase(const Locale(language: 'tr')), 'i');
expect('\u0130'.toLocaleLowerCase(enUS), isNot('i'));
expect('\u0130'.toLocaleLowerCase(Locale.parse('tr'), 'i');
expect('\u0130'.toLocaleLowerCase(Locale.parse('en-US'), isNot('i'));

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
package:intl4x type-infra A repository infrastructure change or enhancement
Projects
None yet
3 participants