Add Fallback '.notdef' Glyph for Improved Cross-Platform TrueType Font Support #1352

spacegaori · 2025-01-24T07:24:58Z

I propose adding a fallback '.notdef' glyph for TrueType fonts. I encountered a similar issue to this one on macOS and Linux when attempting to use a system font, Times New Roman, for Unicode support. A key error was raised because the system fonts were missing the '.notdef' glyph. While the inclusion of this glyph is mandatory for TTF fonts, it is evidently not standardized across platforms. Instead of assuming the glyph’s existence, implementing a fallback option would improve cross-platform support and prevent these errors.

Checklist:

A unit test is covering the code added / modified by this PR
This PR is ready to be merged
In case of a new feature, docstrings have been added, with also some documentation in the docs/ folder
A mention of the change is present in CHANGELOG.md

Lucas-C · 2025-01-24T07:38:35Z

Hi @spacegaori

Thank you very much for this PR 👍

Overall it seems a good idea to me, but I'll let @andersonhc have a look at this, because he's more expert on she subject of fonts than myself 😅

A couple of suggestions of minors changes to make to this PR before we accept it:

produce a warning in this case using LOGGER.warning (you'll find several examples in the code)
add a unit test (we will need a font without .notdef that is free for us to embed in this repo, right?)
add a mention of this to CHANGELOG.md file

… imports

…nt missing this glyph.

andersonhc

Good job on this PR. I really like your solution

andersonhc · 2025-01-25T04:42:04Z

fpdf/fonts.py

+
+            self.ttfont["glyf"][".notdef"] = pen.glyph()
+            self.ttfont["hmtx"][".notdef"] = (600, 0)
+


I believe having the fixed 0,0 to 600,600 would look weird on different fonts.
Maybe having your coordinates set to (ttfont['head'].xMin, ttfont['head'].yMax), (ttfont['head'].xMax, ttfont['head'].yMax) would generate a more consistent notdef outline.

andersonhc · 2025-01-25T04:44:27Z

test/fonts/test_add_font.py

+        "Fallback glyph will be provided."
+    ) in caplog.text
+
+


Times New Roman is a commercial font owned by Monotype, so we can't include it in our repository.

I recommend getting a OFL font and renaming the .notdef glyph to something else to create your test font.

something like this:

from fontTools import ttLib font = ttLib.TTFont("NotoSans-Regular.ttf") glyphnames = font.getGlyphOrder() glyphnames[0] = "dummy" font = ttLib.TTFont("NotoSans-Regular.ttf") font.setGlyphOrder(glyphnames) post = font['post'] font.save("NotoSans-Regular-without-notdef.ttf")

post = font['post']

What does this line do?

post = font['post']

What does this line do?

If you don't read the 'post' table, fontTools will never apply the change on this table and the change won't be saved on the new font. It's because of the lazy loading they do to improve performance.

You don't need to include the "NotoSans-Regular-without-notdef.ttf" - you can create the font and commit the ttf into the repository.

Also, the CI is failing because test/font/test_charmap.py will create a test file with the first 999 chars of every font on test/font, so you need to run it with generate=True once to create a reference file for the new font.

I see. I'll run it once.

You don't need to include the "NotoSans-Regular-without-notdef.ttf" - you can create the font and commit the ttf into the repository.

Do you mean that I don't need to include the process of creating "NotoSans-Regular-without-notdef.ttf" and just include the created font?

If you don't read the 'post' table, fontTools will never apply the change on this table and the change won't be saved on the new font. It's because of the lazy loading they do to improve performance.

Maybe it would be worth adding a comment explaining that?

post = font['post']

FYI, this line causes a linter error.

test/fonts/test_add_font.py:158:4: W0612: Unused variable 'post' (unused-variable)

As you included the TTF font already, I'd say just remove generating the font again (remove lines 151 to 161).

It should be good to merge after that.

… values

andersonhc · 2025-01-29T02:29:16Z

@allcontributors please add @spacegaori for code

allcontributors · 2025-01-29T02:29:25Z

@andersonhc

I've put up a pull request to add @spacegaori! 🎉

spacegaori · 2025-01-29T21:24:05Z

Linter fails on files that are not relevant to this PR 🤔

would reformat /home/runner/work/fpdf2/fpdf2/fpdf/annotations.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/enums.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/image_datastructures.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/line_break.py
would reformat /home/runner/work/fpdf2/fpdf2/scripts/benchmark_borb_vs_fpdf2.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/table.py
would reformat /home/runner/work/fpdf2/fpdf2/test/fonts/test_wraps_zerowidthspace.py
would reformat /home/runner/work/fpdf2/fpdf2/test/test_positioning.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/fpdf.py
would reformat /home/runner/work/fpdf2/fpdf2/test/text/test_varied_fragments.py

Oh no! 💥 💔 💥
10 files would be reformatted, 187 files would be left unchanged.

andersonhc · 2025-01-29T22:10:36Z

Linter fails on files that are not relevant to this PR 🤔

would reformat /home/runner/work/fpdf2/fpdf2/fpdf/annotations.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/enums.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/image_datastructures.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/line_break.py
would reformat /home/runner/work/fpdf2/fpdf2/scripts/benchmark_borb_vs_fpdf2.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/table.py
would reformat /home/runner/work/fpdf2/fpdf2/test/fonts/test_wraps_zerowidthspace.py
would reformat /home/runner/work/fpdf2/fpdf2/test/test_positioning.py
would reformat /home/runner/work/fpdf2/fpdf2/fpdf/fpdf.py
would reformat /home/runner/work/fpdf2/fpdf2/test/text/test_varied_fragments.py

Oh no! 💥 💔 💥
10 files would be reformatted, 187 files would be left unchanged.

I think a new version of Black is changing what it checks. However Lucas already merged #1354 performing the changes. You can just sync or rebase your fork with the new commits and that should be all.

Add fallback '.notdef' glyph

9fd37a8

spacegaori added 3 commits January 24, 2025 16:07

add logging for missing .notdef glyph in TrueType fonts & add missing…

1c005fe

… imports

add test for handling missing .notdef glyph in TrueType fonts. add fo…

a3855aa

…nt missing this glyph.

update CHANGELOG.md

369fbf1

spacegaori marked this pull request as ready for review January 25, 2025 00:17

spacegaori requested a review from gmischler as a code owner January 25, 2025 00:17

andersonhc reviewed Jan 25, 2025

View reviewed changes

spacegaori added 2 commits January 26, 2025 23:42

Cannot include commercial font

83af3e7

Generate a more consistent .notdef outline instead of using arbitrary…

d9a247f

… values

spacegaori force-pushed the master branch from 1f82b86 to ab766ee Compare January 27, 2025 08:17

Modify test to use a test font instead of a commercial font

ab766ee

spacegaori requested a review from andersonhc January 27, 2025 21:25

spacegaori added 4 commits January 27, 2025 20:24

upload a reference notdef font file

06739cb

assign to variable to avoid being a 'pointless-statement'

d14084c

Add a comment explaining lazy loading in fontTools.

ec10cf8

Regenerate test font

8d8d946

allcontributors bot mentioned this pull request Jan 29, 2025

add spacegaori as a contributor for code #1353

Merged

Remove test font generation script within test

65af26f

spacegaori force-pushed the master branch from fd1ff2c to 65af26f Compare January 29, 2025 05:57

spacegaori added 2 commits January 29, 2025 14:35

Merge branch 'py-pdf:master' into master

b25f555

Remove unused TTFont import

011254c

andersonhc merged commit d07a6a7 into py-pdf:master Jan 29, 2025
16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Fallback '.notdef' Glyph for Improved Cross-Platform TrueType Font Support #1352

Add Fallback '.notdef' Glyph for Improved Cross-Platform TrueType Font Support #1352

spacegaori commented Jan 24, 2025 •

edited

Loading

Lucas-C commented Jan 24, 2025

andersonhc left a comment

andersonhc Jan 25, 2025

andersonhc Jan 25, 2025

spacegaori Jan 27, 2025 •

edited

Loading

andersonhc Jan 28, 2025

andersonhc Jan 28, 2025

spacegaori Jan 28, 2025

Lucas-C Jan 28, 2025

spacegaori Jan 28, 2025

andersonhc Jan 29, 2025

andersonhc commented Jan 29, 2025

allcontributors bot commented Jan 29, 2025

spacegaori commented Jan 29, 2025

andersonhc commented Jan 29, 2025


		self.ttfont["glyf"][".notdef"] = pen.glyph()
		self.ttfont["hmtx"][".notdef"] = (600, 0)

Add Fallback '.notdef' Glyph for Improved Cross-Platform TrueType Font Support #1352

Add Fallback '.notdef' Glyph for Improved Cross-Platform TrueType Font Support #1352

Conversation

spacegaori commented Jan 24, 2025 • edited Loading

Lucas-C commented Jan 24, 2025

andersonhc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

spacegaori Jan 27, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andersonhc commented Jan 29, 2025

allcontributors bot commented Jan 29, 2025

spacegaori commented Jan 29, 2025

andersonhc commented Jan 29, 2025

spacegaori commented Jan 24, 2025 •

edited

Loading

spacegaori Jan 27, 2025 •

edited

Loading