diff --git a/.gitignore b/.gitignore index 92d916bc4e..6acb4a9dbd 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ .thumb.png layers/ data/ +fonts/ tmp/ *.xml *.osm.pbf diff --git a/INSTALL.md b/INSTALL.md index bc73b0a6d5..74ae092974 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -55,49 +55,16 @@ The script downloads shapefiles, loads them into the database and sets up the ta ## Fonts The stylesheet uses Noto, an openly licensed font family from Google with support for multiple scripts. The stylesheet uses Noto's "Sans" style where available. If not available, this stylesheet uses another appropriate style of the Noto family. The "UI" version is used where available, with its vertical metrics which fit better with Latin text. -DejaVu Sans is used as an optional fallback font for systems without Noto Sans. If all the Noto fonts are installed, it should never be used. Noto Naskh Arabic UI is used an an optional fallback font for systems without Noto Sans Arabic. - Hanazono is used a fallback for seldom used CJK characters that are not covered by Noto. -Unifont is used as a last resort fallback, with it's excellent coverage, common presence on machines, and ugly look. For compatibility reasons, we support two Linux-distributions-specific versions of Unifont, therefore it's expected that you *always* get a warning about a missing Unifont version. - -If you do not install all the fonts, the rendering itself will not break, but missing glyphs will be ugly. - For more details, see the documentation at [fonts.mss](style/fonts.mss). -### Installation on Ubuntu/Debian - -On Ubuntu 16.04 or Debian Testing you can download and install most of the required fonts +To download the fonts, run the following script ``` -sudo apt-get install fonts-noto-cjk fonts-noto-hinted fonts-noto-unhinted fonts-hanazono ttf-unifont +scripts/get-fonts.sh ``` -Noto Emoji Regular (*not* Noto Color Emoji) can be downloaded [from the Noto Emoji repository](https://github.com/googlefonts/noto-emoji). - -It might be useful to have a more recent version of the fonts for [rare non-latin scripts](#non-latin-scripts). The current upstream font release has also some more scripts and style variants than in the Ubuntu package. It can be installed [from source](https://github.com/googlefonts/noto-fonts/blob/master/FAQ.md#where-are-the-fonts). - -DejaVu is packaged as `fonts-dejavu-core`. - -### Installation on other operating systems - -The fonts can be downloaded here: - -* [Noto homepage](https://www.google.com/get/noto/) and [Noto github repositories](https://github.com/googlefonts?utf8=%E2%9C%93&q=noto) -* [DejaVu homepage](https://dejavu-fonts.org/) -* [Hanazono homepage](http://fonts.jp/hanazono/) -* [Unifont homepage](http://unifoundry.com/) - -After the download, you have to install the font files in the usual way of your operating system. - -### Non-latin scripts - -For proper rendering of non-latin scripts, particularly those with complicated diacritics and tone marks the requirements are - -* FreeType 2.6.2 or later for CJK characters - -* A recent enough version of Noto with coverage for the scripts needed. - ## Dependencies For development, a style design studio is needed. @@ -128,9 +95,3 @@ For deployment, CartoCSS and Mapnik are required. * [Mapnik](https://github.com/mapnik/mapnik/wiki/Mapnik-Installation) >= 3.0 With CartoCSS you compile these sources into a Mapnik compatible XML file. When running CartoCSS, specify the Mapnik API version you are using (at least 3.0.0: `carto -a "3.0.0"`). - -If you're calling Mapnik in your own program, remember to load the XML file in non strict mode. This way, fonts declared with alternative names will only generate warnings, not errors. For instance, using the Python bindings, this becomes: - -```python -mapnik.load_map(mapnik.Map(width, height), xml_filename, False) # False for non-strict mode -``` diff --git a/scripts/docker-startup.sh b/scripts/docker-startup.sh index 4d2d15f6e3..c3177ee9d1 100644 --- a/scripts/docker-startup.sh +++ b/scripts/docker-startup.sh @@ -54,6 +54,9 @@ EOF # Downloading and importing needed shapefiles scripts/get-external-data.py $EXTERNAL_DATA_SCRIPT_FLAGS + + # Download fonts + scripts/get-fonts.sh ;; kosmtik) diff --git a/scripts/get-fonts.sh b/scripts/get-fonts.sh new file mode 100755 index 0000000000..710a9c1a55 --- /dev/null +++ b/scripts/get-fonts.sh @@ -0,0 +1,133 @@ +#!/bin/sh +set -e + +FONTDIR="./fonts" + +mkdir -p "${FONTDIR}" + +# download filename url +download() { + ## Download if newer, and if curl fails, clean up and exit + curl --fail --compressed -A "get-fonts.sh/osm-carto" -o "$1" -z "$1" -L "$2" || { echo "Failed to download $1 $2"; rm -f "$1"; exit 1; } +} + +# TTF Hinted Noto Fonts + +# Fonts available in regular, bold, and italic +REGULAR_BOLD_ITALIC="NotoSans" + +# Fonts available in regular and bold +REGULAR_BOLD="NotoSansAdlam +NotoSansAdlamUnjoined +NotoSansArabicUI +NotoSansArmenian +NotoSansBalinese +NotoSansBamum +NotoSansBengaliUI +NotoSansCanadianAboriginal +NotoSansCham +NotoSansCherokee +NotoSansDevanagariUI +NotoSansEthiopic +NotoSansGeorgian +NotoSansGujaratiUI +NotoSansGurmukhiUI +NotoSansHebrew +NotoSansJavanese +NotoSansKannadaUI +NotoSansKayahLi +NotoSansKhmerUI +NotoSansLaoUI +NotoSansLisu +NotoSansMalayalamUI +NotoSansMyanmarUI +NotoSansOlChiki +NotoSansOriyaUI +NotoSansSinhalaUI +NotoSansSundanese +NotoSansSymbols +NotoSansTaiTham +NotoSansTamilUI +NotoSansTeluguUI +NotoSansThaana +NotoSansThaiUI +NotoSerifTibetan" + +# Fonts with regular and black, but no bold +REGULAR_BLACK="NotoSansSyriac" + +# Fonts only available in regular +REGULAR="NotoSansBatak +NotoSansBuginese +NotoSansBuhid +NotoSansChakma +NotoSansCoptic +NotoSansHanunoo +NotoSansLepcha +NotoSansLimbu +NotoSansMandaic +NotoSansMongolian +NotoSansNewTaiLue +NotoSansNKo +NotoSansOsage +NotoSansOsmanya +NotoSansSamaritan +NotoSansSaurashtra +NotoSansShavian +NotoSansSymbols2 +NotoSansTagalog +NotoSansTagbanwa +NotoSansTaiLe +NotoSansTaiViet +NotoSansTifinagh +NotoSansVai +NotoSansYi" + +# Download the fonts in the lists above + +for font in $REGULAR_BOLD_ITALIC; do + regular="$font-Regular.ttf" + bold="$font-Bold.ttf" + italic="$font-Italic.ttf" + download "${FONTDIR}/${regular}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${regular}" + download "${FONTDIR}/${bold}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${bold}" + download "${FONTDIR}/${italic}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${italic}" +done + +for font in $REGULAR_BOLD; do + regular="$font-Regular.ttf" + bold="$font-Bold.ttf" + download "${FONTDIR}/${regular}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${regular}" + download "${FONTDIR}/${bold}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${bold}" +done + +for font in $REGULAR_BLACK; do + regular="$font-Regular.ttf" + black="$font-Black.ttf" + download "${FONTDIR}/${regular}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${regular}" + download "${FONTDIR}/${black}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${black}" +done + +for font in $REGULAR; do + regular="$font-Regular.ttf" + download "${FONTDIR}/${regular}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${regular}" +done + +# Other noto fonts which don't follow the URL pattern above +download "${FONTDIR}/NotoSansCJKjp-Regular.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/Japanese/NotoSansCJKjp-Regular.otf" +download "${FONTDIR}/NotoSansCJKjp-Bold.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/Japanese/NotoSansCJKjp-Bold.otf" + +# Fonts in zipfiles need a temporary directory +TMPDIR=$(mktemp -d -t get-fonts.XXXXXXXXX) +trap "rm -rf ${TMPDIR} ${FONTDIR}/static" EXIT + +# Noto Emoji B&W isn't available as a separate download, so we need to download the package and unzip it +curl --fail -A "get-fonts.sh/osm-carto" -o "${TMPDIR}/Noto_Emoji.zip" -L 'https://fonts.google.com/download?family=Noto%20Emoji' + +unzip -oqq "${TMPDIR}/Noto_Emoji.zip" static/NotoEmoji-Regular.ttf static/NotoEmoji-Bold.ttf -d "${FONTDIR}" +mv "${FONTDIR}/static/NotoEmoji-Regular.ttf" "${FONTDIR}" +mv "${FONTDIR}/static/NotoEmoji-Bold.ttf" "${FONTDIR}" + +curl --fail -A "get-fonts.sh/osm-carto" -o "${TMPDIR}/hanazono.zip" -L 'https://osdn.net/frs/redir.php?f=hanazono-font%2F68253%2Fhanazono-20170904.zip' + +unzip -oqq "${TMPDIR}/hanazono.zip" HanaMinA.ttf HanaMinB.ttf -d "${FONTDIR}" diff --git a/style/fonts.mss b/style/fonts.mss index 102702519a..47b94c08d3 100644 --- a/style/fonts.mss +++ b/style/fonts.mss @@ -2,81 +2,56 @@ About fonts: Noto is a font family that wants to cover most of Unicode with a harmonic -design across various scripts. We use Noto for most text, with some support -for backward-compatibility and some fallback fonts. +design across various scripts. We use Noto for most text and some fallback fonts. By order: 1. Noto Sans is available for most scripts and it is used as a first choice. Where available the UI version of the fonts – which provides tighter vertical -metrics – is used (except for the base font, where the UI version is deprecated -since Noto Phase III, and Sinhala where both versions are used for backwards -compatibility with Ubuntu 16.04). We intent to have all scripts of Noto in +metrics – is used. We intent to have all scripts of Noto in our list except dead (historic) scripts of whom we assume that they are not used in “name” tags in OSM. Most of the list is in alphabetical order, but there are some exceptions. - Noto Sans is before all other fonts - - The CJK fonts are manually ordered. The used CJK font covers all CJK - languages, but defaults to the japanese glyph style if various glyph - styles are available. We have to default to one of JP, KR, SC, TC because - this carto style has no knowledge about what language the “names” tag - contains. As in Korea Han characters are not so widely used, it seems - better to default to either Chinese or Japanese. As Chinese exists in the - two variants SC/TC, it won’t be a uniform rendering anyway. So we default - to Japanese. However, this choice stays somewhat arbitrary and subjective. - See also https://github.com/gravitystorm/openstreetmap-carto/issues/2208 - - For the Syriac script, there exist Noto Sans Syriac Eastern, - Noto Sans Syriac Western Regular and Noto Sans Syriac Estrangela Regular. - As for CJK fonts, we have no knowledge about what language the “names” tag - contains. We choose Syriac Eastern because it seems to be the most - widespreaded script variant (Syriac Western is less common, and Syriac - Estrangela mostly of historic interest.) - -2. Noto provides various variants of Arabic: Noto Kufi Arabic, Noto Naskh -Arabic, Noto Nastaliq Urdu and Noto Sans Arabic. Kufi and Urdu styles are not -widespread in use. Noto Sans Arabic (a Naskh-style low-contrast “Sans” font) and -Noto Naskh Arabic are the fonts with the greatest coverage and provide an UI -variant. This style uses Noto Sans Arabic UI because it’s consistent with the -other Sans fonts and legible. The Arabic fonts are placed behind Sans fonts -because they might re-define some commonly used signs like parenthesis or -quotation marks, and the arabic design should not overwrite the standard design. -The list still includes Noto Naskh Arabic UI for compatibility on systems -without Noto Sans Arabic UI. - -3. Noto Serif is used for scripts that are not supported by Noto Sans. Currently + + - Noto Arabic and Noto CJK JP are placed behind sans fonts because commonly + used signs like parenthesis or quotation marks, and their design should + not overwrite the standard design. + +3. The used CJK font covers all CJK languages, but defaults to the Japanese +glyph style if various glyph styles are available. We have to default to one +of JP, KR, SC, TC because this style has no knowledge about what language the +“names” tag contains. As in Korea Han characters are not so widely used, it is +better to default to either Chinese or Japanese. As Chinese exists in the two +variants SC/TC, it won’t be a uniform rendering anyway. So we default to +Japanese. However, this choice stays somewhat arbitrary and subjective. + +4. Similar to CJK Han Unification, there are multiple scripts for Arabic +scripts, and the style has no way to know what language the text is in. This +style uses Noto Sans Arabic UI because it’s consistent with the other Sans +fonts and legible. + +5. Noto Serif is used for scripts that are not supported by Noto Sans. Currently this is only Tibetan: The old Noto Sans Tibetan has been renamed to Noto Serif -Tibetan in 2015, since then only Noto Serif Tibetan gets updated. We still -include the old one for compatibility on systems without Noto Serif Tibetan. -See also https://github.com/googlefonts/noto-fonts/issues/1540 +Tibetan in 2015, since then only Noto Serif Tibetan gets updated. -4. Noto provides two variants of Emoji: Noto Color Emoji and Noto Emoji. The +6. Noto provides two variants of Emoji: Noto Color Emoji and Noto Emoji. The colour variant is a SVG flavoured OpenType font that contains coloured emojis. This is not useful in cartography, so we use the “normal” monochromatic Noto Emoji. -5. The list still includes DejaVu for compatibility on systems without Noto. - -6. Fallback fonts. Hanazono covers almost all CJK characters, even in Unicode -Plane 2. Unifont is a fallback of last resort with full coverage in Plane 0 -(Unifont Medium), some coverage in Plane 1 (Unifont Upper Medium) and no -coverage in Plane 2. Unifont has different font names on different Linux -distributions (and sometimes even in different versions of the same Linux -distribution), see #429 and #2924 for details. We have therefore both, -“Unifont Medium” and “unifont Medium” in our list, so on almost all -distributions we catch the font if installed – and you will always get an error -message for the other missing font name. We prefer having an error message than -requiring our users to customize the font list depending on their Linux -distribution. Unifont Medium Sample would cover the BMP PUA with -replacement characters, but cannot be used because Mapnik does not -support SBIT TTF. +7. Fallback fonts. Hanazono covers almost all CJK characters, even in Unicode +Plane 2. */ +/* Use vendored fonts. This allows for more recent versions and better coverage */ +font-directory: url('fonts'); + /* A regular style. */ @book-fonts: "Noto Sans Regular", - "Noto Sans CJK JP Regular", "Noto Sans Adlam Regular", "Noto Sans Adlam Unjoined Regular", "Noto Sans Armenian Regular", @@ -119,11 +94,11 @@ A regular style. "Noto Sans Samaritan Regular", "Noto Sans Saurashtra Regular", "Noto Sans Shavian Regular", - "Noto Sans Sinhala UI Regular", "Noto Sans Sinhala Regular", + "Noto Sans Sinhala UI Regular", "Noto Sans Sundanese Regular", "Noto Sans Symbols Regular", "Noto Sans Symbols2 Regular", - "Noto Sans Syriac Eastern Regular", + "Noto Sans Syriac Regular", "Noto Sans Tagalog Regular", "Noto Sans Tagbanwa Regular", "Noto Sans Tai Le Regular", @@ -137,25 +112,27 @@ A regular style. "Noto Sans Vai Regular", "Noto Sans Yi Regular", - "Noto Sans Arabic UI Regular", "Noto Naskh Arabic UI Regular", + "Noto Sans CJK JP Regular", + "Noto Sans Arabic UI Regular", - "Noto Serif Tibetan Regular", "Noto Sans Tibetan Regular", + "Noto Serif Tibetan Regular", "Noto Emoji Regular", - "DejaVu Sans Book", - - "HanaMinA Regular", "HanaMinB Regular", - "Unifont Medium", "unifont Medium", "Unifont Upper Medium"; + "HanaMinA Regular", "HanaMinB Regular"; /* A bold style is available for almost all scripts. Bold text is heavier than regular text and can be used for emphasis. Fallback is a regular style. */ @bold-fonts: "Noto Sans Bold", - "Noto Sans CJK JP Bold", + "Noto Sans Adlam Bold", + "Noto Sans Adlam Unjoined Bold", "Noto Sans Armenian Bold", + "Noto Sans Balinese Bold", + "Noto Sans Bamum Bold", "Noto Sans Bengali UI Bold", + "Noto Sans Canadian Aboriginal Bold", "Noto Sans Cham Bold", "Noto Sans Cherokee Bold", "Noto Sans Devanagari UI Bold", @@ -164,24 +141,31 @@ regular text and can be used for emphasis. Fallback is a regular style. "Noto Sans Gujarati UI Bold", "Noto Sans Gurmukhi UI Bold", "Noto Sans Hebrew Bold", + "Noto Sans Javanese Bold", "Noto Sans Kannada UI Bold", + "Noto Sans Kayah Li Bold", "Noto Sans Khmer UI Bold", "Noto Sans Lao UI Bold", + "Noto Sans Lisu Bold", "Noto Sans Malayalam UI Bold", "Noto Sans Myanmar UI Bold", + "Noto Sans Ol Chiki Bold", "Noto Sans Oriya UI Bold", - "Noto Sans Sinhala UI Bold", "Noto Sans Sinhala Bold", + "Noto Sans Sinhala UI Bold", + "Noto Sans Sundanese Bold", "Noto Sans Symbols Bold", + "Noto Sans Syriac Black", + "Noto Sans Tai Tham Bold", "Noto Sans Tamil UI Bold", "Noto Sans Telugu UI Bold", "Noto Sans Thaana Bold", "Noto Sans Thai UI Bold", - "Noto Sans Arabic UI Bold", "Noto Naskh Arabic UI Bold", - - "Noto Serif Tibetan Bold", "Noto Sans Tibetan Bold", + "Noto Sans CJK JP Bold", + "Noto Sans Arabic UI Bold", - "DejaVu Sans Bold", + "Noto Serif Tibetan Bold", + "Noto Emoji Bold", @book-fonts;