Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Noto support #2349

Merged
merged 14 commits into from
Sep 19, 2016
Merged

Initial Noto support #2349

merged 14 commits into from
Sep 19, 2016

Conversation

sommerluk
Copy link
Collaborator

@sommerluk sommerluk commented Sep 15, 2016

Uses Noto as default font.
Fixes #2338
Fixes #2337
Fixes #2336
Fixes #2240
Fixes #2204
Fixes #1335 (only if DejaVu is dropped)
Fixes #1067
Fixes #677
Fixes #486
Fixes #362
Fixes #294

We loose the oblique style for Tamil and Thai, but we gain the bold style for almost all scripts.

Noto covers most scripts of the world “with a harmonious look and feel”.

Furthermore, the Unifont list is extended (added Unifont Upper).

TODO:

  • What are the performance implications of such a long font list?
  • Can some fonts be removed because they correspond to "dead" languages not used on the ground?
  • Can some fonts be removed because other Noto fonts cover all the glyphs they have?
  • Is the ordering right on bold with Noto Kufi Arabic Bold before the regular fonts?
  • Is the way we're doing CJK right? If Noto Sans CJK JP Regular has all the glyphs, the other CJK fonts will never be used. The Noto Guidelines mention there is both "Noto Sans CJK {JP, KR, SC, TC}" and "Noto Sans {JP, KR, SC, TC}", what's the difference?
  • Update INSTALL.md

@sommerluk
Copy link
Collaborator Author

The list contains all Sans fonts of the current Noto release. For Arabic it uses Noto Kufi (I don’t know much about Arabic). For Emoji is uses the monochromatic Noto version. Noto Serif and Noto Mono are not used (is covered yet by Noto Sans).

Only one of the CJK fonts is necessary.

Only either the UI font or the non-UI font is necessary. If both are available, the UI font is prefferred.

The list of bold Noto fonts is the same as for regular Noto fonts. While not all of them are yet currently available, the next release of Noto will add many new bold fonts. Fallback is the regular font.

The previously used fonts are still in the list (at the end), so someone who updates the style and does not have Noto will get the same result as in previous versions of the style.

Documentation still needs to be updated (INSTALL.md…)

@sommerluk
Copy link
Collaborator Author

Screenshots from South Korea/Seoul

Left: before.

Right: after.

screenshot 1
screenshot 2
screenshot 3
screenshot 4
screenshot 5
screenshot 6
screenshot 7
screenshot 8
screenshot 9
screenshot 10

@HolgerJeromin
Copy link
Contributor

fixes #2265, too?

to remain backward-compatible.

5. Unifont. This is a simple and not really beautiful font, but it has an
extremely great coverage. Unifont Medium covers the hole Unicode BMP without
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/hole/whole.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've called unifont a "fallback of last resort" which is a good explanation for it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

"Noto Sans Ugaritic Bold", "Noto Sans Vai Bold",
"Noto Sans Yi Bold",
"Noto Kufi Arabic Bold", "Noto Emoji Bold",
"Noto Sans UI Regular", "Noto Sans Regular",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We duplicated the regular fonts list into the bold list because for the specific fonts we were using there were some that had regular glyphs but not bold. Is this also the case with Noto? If not, we can drop the duplicates.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know this is not the case with Noto, so we can drop the duplicates.

“Noto Sans CJK *” fonts, because the CJK fonts contain non-Noto latin characters
that should not be used. Noto Serif and Noto Mono are available for various
(mostly latin-like) scripts. They are ignored here in favour of the respective
Noto Sans variant. For fonts that are known to have an UI variant (a variant
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can cut the Serif and Mono explanation. We don't ignore them in favour of a Noto Sans variant, it's that we don't use serif or mono fonts in the stylesheet so have no need for them.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

By order:

1. Noto Sans is available for most scripts and it is used here. The list in is
alphabetical order. Note that the “Noto Sans” base font MUST be before the
Copy link
Collaborator

@pnorman pnorman Sep 15, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is alphabetical order the best way to do this? We say that it's not alphabetical by putting CJK and Kufi later.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first group is Noto Sans. Within this group, there is indeed alphabetical order (including CJK). For CJK the requierement is that Noto Sans is before it, and this is the case in alphabetical order.

Noto Kufi is not Sans. It’s a group of itself.

Noto Emoji.

4. The list still includes the previously used font selection of this style
to remain backward-compatible.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're happy with Noto we can cut out the previous list. I might still keep DejaVu around for compatibility, but not the others.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, but I see I omitted to fix the comment.

Copy link
Collaborator

@pnorman pnorman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few higher level considerations here. GitHub's image hosting is down at the time I'm writing it so I'll come back and add more detail later

  • Do we want to replace DejaVu Sans with Noto Sans? This could have a wide change on the look of the style
  • Is the choice of UI variants over normal one we want to make?
  • If there is both a UI variant and a normal one are there any glyphs that the normal one has but not the UI?
  • What are the performance implications of such a long font list?
    • Can some fonts be removed because they correspond to "dead" languages not used on the ground?
    • Can some fonts be removed because other Noto fonts cover all the glyphs they have?
  • Is the ordering of CJK fonts one we are happy with?

@pnorman
Copy link
Collaborator

pnorman commented Sep 15, 2016

I just want to note that I'm trying pull request reviews (#2348) here

@sommerluk
Copy link
Collaborator Author

sommerluk commented Sep 15, 2016

Do we want to replace DejaVu Sans with Noto Sans? This could have a wide change on the look of the style

I would recommend this. It will be anyway a wide change if we keep DejaVu, so I would use this ocasion to get a consistent set of fonts.

Is the choice of UI variants over normal one we want to make?

When playing around, I could not find a difference between the glyphs of UI and the standard variant. The only difference seems to be the vertical default line spacing, which is smaller for the UI variant, and the Noto documentation claims that they are designed for user interfaces with restricted vertical space. This might be useful, because often there is not much place on the map. I would therefor preffer the UI variant.

If there is both a UI variant and a normal one are there any glyphs that the normal one has but not the UI?

As far as I know: No. So the normal one could be dropped.

What are the performance implications of such a long font list?

Experience: When installing a missing font while Kosmtik was running, Kosmtik was not able to render with the new font until Kosmtik was terminated and re-started. So maybe when a style is loaded, the fonts are searched only at the beginning, and not-installed fonts of the list are not searched again?

Can some fonts be removed because they correspond to "dead" languages not used on the ground?

If this is deisred, yes.

Can some fonts be removed because other Noto fonts cover all the glyphs they have?

Apart from the yet discussed bold/regular-fallback and UI/non-UI duplicates: All but one of the four CJK fonts – they have all the same glyph coverage. While this are only three font entries less, this can save a lot of disk space, because the CJK fonts are the biggest ones.

Is the ordering of CJK fonts one we are happy with?

  • In Korea Han seems to be used seldom. They use mostly Hangul, so Han Unification is not such a big problem for Korea. See What about Han unification? #2208 (comment) We would not gain much with CJK KR as default. So CJK KR would be a bad choise for a default.
  • So the default should be either Chinese (simplified or traditional) or Japanese. Looking around in the Internet, it seems that for Japanese people it is more important to have Japanese signs and not Chinese signs, while Chinese people are more tolerant to Japanese signs (maybe also because with “simplified” and “traditional” there are yet two parallel versions within their own language). I would use the Japanese variant as default.

@sommerluk
Copy link
Collaborator Author

sommerluk commented Sep 15, 2016

fixes #2265, too?

Not exactly. But it would at least make the problem in #2265 less visible. And indeed there is no evidently perfect solution for #2265 so far…

@pnorman
Copy link
Collaborator

pnorman commented Sep 16, 2016

Experience: When installing a missing font while Kosmtik was running, Kosmtik was not able to render with the new font until Kosmtik was terminated and re-started. So maybe when a style is loaded, the fonts are searched only at the beginning, and not-installed fonts of the list are not searched again?

I'm not so worried about map file load time and checking for fonts, it's rendering a particular label where the glyphs aren't found in the first fonts: #414 (comment)

@pnorman
Copy link
Collaborator

pnorman commented Sep 17, 2016

I'm having a hard time getting system fonts working with Kosmtik so haven't been able to review yet.

@pnorman
Copy link
Collaborator

pnorman commented Sep 17, 2016

Testing with 18/41562/89786

Deja Vu

Noto Sans UI

Noto Sans

These fonts shouldn't be used anymore with Noto. DejaVu is kept in
for development purposes.
@pnorman
Copy link
Collaborator

pnorman commented Sep 17, 2016

Note: Approval of changes was investigating GitHub reviews, not everything is done yet.

@pnorman
Copy link
Collaborator

pnorman commented Sep 17, 2016

10
image

12
image

14
image

18
image

Rendered in Word 2010 on Windows 8.1 with fonts from fonts-noto git repo

@pnorman
Copy link
Collaborator

pnorman commented Sep 17, 2016

Noto Sans and Noto Sans UI are the same on latin glyphs, which is why my tests look pixel for pixel identical.

A "tall" script shows the difference much better

image

Worth noting that https://material.google.com/style/typography.html has a bunch of recommendations on font sizes, line height, and tracking.

@pnorman
Copy link
Collaborator

pnorman commented Sep 17, 2016

Let's go with UI fonts as the only choice if they have the coverage in the language. Worth noting that the download zip doesn't have UI versions, but that doesn't impact the fonts coming from packages (notofonts/noto-fonts#744).

Not every Noto font is available in Bold.
@pnorman
Copy link
Collaborator

pnorman commented Sep 17, 2016

The list of bold Noto fonts is the same as for regular Noto fonts. While not all of them are yet currently available, the next release of Noto will add many new bold fonts. Fallback is the regular font.

I missed that before cleaning up the list of fonts. I'm inclined to leave it with my changes because we'll need to check the glyph counts at release.

Remaining issues

  • [] What are the performance implications of such a long font list?
  • [] Can some fonts be removed because they correspond to "dead" languages not used on the ground?
  • [] Can some fonts be removed because other Noto fonts cover all the glyphs they have?
  • [] Is the ordering right on bold with Noto Kufi Arabic Bold before the regular fonts?
  • [] Is the way we're doing CJK right? If Noto Sans CJK JP Regular has all the glyphs, the other CJK fonts will never be used. The Noto Guidelines mention there is both "Noto Sans CJK {JP, KR, SC, TC}" and "Noto Sans {JP, KR, SC, TC}", what's the difference?

@sommerluk
Copy link
Collaborator Author

Candidates for elemination (based on a quick wikipedia search for completly dead languages):

Avestan
Brahmi
Carian
Cuneiform
Cypriot
Deseret
Egyptian Hieroglyphs
Glagolitic
Gothic
Imperial Aramaic
Inscriptional Pahlavi
Inscriptional Parthian
Kaithi
Kharoshthi
Linear B
Lycian
Lydian
Meetei Mayek
Ogham
Old Italic
Old Persioan
Old South Arabian
Old Turkic
Phags Pa
Phoenician
Rejang
Runic
Syloti Nagri
Ugaritic

@sommerluk
Copy link
Collaborator Author

About performance: Wikipedia has at https://en.wikipedia.org/wiki/List_of_writing_systems#List_of_writing_scripts_by_adoption a list of writing systems ordered by number of people who use it. Maybe we want to order our list like this?

@sommerluk
Copy link
Collaborator Author

"Noto Sans CJK {JP, KR, SC, TC}" and "Noto Sans {JP, KR, SC, TC}", what's the difference?

The former contains all available CJK glyphs in all language variants. Whenever a glyph has various language variants, than {JP, KR, SC, TC} is the default one. The other variants are available with the OpenType LOCL feature.

The latter does not contain all available CJK glyphs, but only the subset that is necessary for the choosen language {JP, KR, SC, TC}. That means that less characters/codepoints are covered, and for the covered characters, it does not contain any alternative LOCL variants.

@imagico
Copy link
Collaborator

imagico commented Sep 19, 2016

Since the rendering of Arabic is very different it would probably be good to get some feedback from routine Arabic readers on the matter. To me it seems it might be significantly better readable at small font sizes but on the other hand the proportions of the characters are so very different from common fonts that it might require a lot to get used to.

@neuhausr
Copy link

Looking at the Japanese example, I agree with @pnorman 's comment that the new version almost looks bold. In many cases, that's an improvement, but for more complex kanji characters, it makes the strokes run together and thus be harder to read.

@pnorman
Copy link
Collaborator

pnorman commented Sep 19, 2016

Since the rendering of Arabic is very different it would probably be good to get some feedback from routine Arabic readers on the matter. To me it seems it might be significantly better readable at small font sizes but on the other hand the proportions of the characters are so very different from common fonts that it might require a lot to get used to.

I couldn't find an Arabic reader, but I had a look at some online Egyptian maps and road navigation signs and the font metrics were closer to what this PR does than traditional writing.

@pnorman pnorman merged commit 71aa1c9 into gravitystorm:master Sep 19, 2016
@matthijsmelissen
Copy link
Collaborator

Thanks, great work @sommerluk!

@notxyz27
Copy link

image
image
Chinese characters look good to me. However, in the second image, the character for 门 is weird. I found that this weird character also shows up when I type 门 using Simplified Chinese input in Chrome address bar but the correct character shows up when I type in search box. I'm kind of sure the weird character is not traditional Chinese.

@sommerluk
Copy link
Collaborator Author

sommerluk commented Sep 20, 2016

@notxyz27 Thanks for reporting this.

The character that you use in your description is “U+95E8 CJK UNIFIED IDEOGRAPH-95E8 : mén” (Unihan at http://www.unicode.org/cgi-bin/GetUnihanData.pl?codepoint=95E8 )

In the attached PDF Dokument.pdf you can see this character (and its semantic counterpart U+9580) in all available Noto glyph variants and in Unifont. What you expect to see is probably number 3. But this style is currently rendering number 1.

In a nutshell: The character is a unified ideograph with different glyph variants. As openstreetmap-carto does not support region-specific rendering and has no knowledge about language in which a label is, it’s not possible to really determine the correct glyph variant. Currently, in case of conflict this style defaults simply to the Japanese variant. This is, indeed, a somewhat arbitrary choise, but we are forced to make a some choise.

In all details: #2208

I fear there is not much we can do about this issue.

@Klumbumbus
Copy link

I don't know if this is relevant to carto rendering. Noto has problems with hinting, which results in unsharp rendering at font sizes 10 and smaller. Atleast it was this way in January. I don't know if this was improved meanwhile. The issue was closed without a solution: notofonts/noto-fonts#556

@thejeshgn
Copy link

Can you share the screenshot with Kannada language? name:kn

@pnorman
Copy link
Collaborator

pnorman commented Sep 23, 2016

Can you share the screenshot with Kannada language? name:kn

Can you link to an area with the name tag in Kannada?

@thejeshgn
Copy link

@pnorman

Here
http://www.openstreetmap.org/#map=12/12.9550/77.6052

The map below displays Kannada labels using name:kn this will give you an idea.

https://thejeshgn.github.io/tangrams-kannada/#12/12.9550/77.6052

Thank you.

@pnorman
Copy link
Collaborator

pnorman commented Sep 23, 2016

The map below displays Kannada labels using name:kn this will give you an idea.

We render the name tag, not name:kn.

@thejeshgn
Copy link

Ah okay.

@dieterdreist

This comment was marked as off-topic.

@stephankn

This comment was marked as off-topic.

@StyXman

This comment was marked as off-topic.

@SomeoneElseOSM

This comment was marked as off-topic.

@kocio-pl

This comment was marked as off-topic.

@dieterdreist

This comment was marked as abuse.

Repository owner locked and limited conversation to collaborators Sep 23, 2016
@sommerluk sommerluk deleted the noto01 branch October 3, 2016 18:41
Repository owner unlocked this conversation Jun 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment