Loading Custom Fonts

[CaseyB]

What tool are you using to create your bitmap fonts? I found one that creates the png and the fnt file but the fnt file is just xml whereas the ones in the examples folder are binary. I tried loading the one I made and I get an error that says: ### Error: Invalid BMF header but I've not been able to find any information online about what a BMF header is.

[andycarle]

@CaseyB We typically use a tool called Glyph Designer. Check out the Creating fonts for applications built on the Moddable SDK document for more details.

 - Andy

[CaseyB]

Thanks for the info. Just an update in case anyone else has the same question. I found an open-source bitmap font maker called FontBuilder and I wrote a little tool that translates the FontBuilder output into the BMFont format. You can find that on my Bitbucket

[andycarle]

@CaseyB Very interesting... Out of curiosity, are you building FontBuilder from source with qmake? Or did you find a source for updated FontBuilder binaries? I'd love to be able to point people to a free font tool, but asking everyone to get a Qt environment up and running for it is probably too much to ask. :)

[CaseyB]

I just pulled the most recent release. It's old but still worked on my Mac.

[andycarle]

Ah, I see. I was trying out the Windows release of that same version, which does not have Sparrow output support.

[wilberforce]

@andycarle
I found this one, targets for win/lin:
https://github.com/vladimirgamalyan/fontbm

in windows I'm using:

F:\fontbm>fontbm --font-file C:\Windows\Fonts\Arial.ttf --output arial-regular-32 --chars-file chars.txt --color 0,0,0 --font-size 32--texture-height 96 --data-format bin

chars.txt:

°0123456789.:

This allows the wanted chars to be added to a file.

[phoddie]

Hey, that looks great. Thank you!

We'll have to give it a try and update the docs. It might also help solve a problem with an upcoming project we're working on.....

[wilberforce]

1. The png file generated gets a paging number that in our case needs to be removed, e.g.:
   arial-regular-32_0.png

https://github.com/vladimirgamalyan/fontbm/blob/master/src/App.cpp#L169

2. The sizing defaults to 512x512 so the final png is not cropped to the minimal size - so the final png has extra transparency that makes the png slightly larger - I'm not sure if thus means the final .bf4 file is larger or not.

If this can be make to compile on the mac, then you'll have a cross platform tool!

[phoddie]

The lack of a Mac version is definitely a headache. They have a Linux build so Mac shouldn't be so bad.

The image dimensions don't matter when using compressed fonts (which is the default).

[andycarle]

@wilberforce That's a great find—thank you for the pointer!

@phoddie For what it's worth, I was able to get fontbm building and running on macOS with no trouble. On a mac that is already set up for our ESP32 builds (i.e. has a modern CMake and Homebrew installed) you just need to:

brew install freetype
git clone https://github.com/vladimirgamalyan/fontbm.git
cd fontbm
cmake .
make

[wilberforce]

Excellant - so we have a cross platform font creation tool!

[phoddie]

So it seems! I was able to build and run following @andycarle's instructions. I haven't tried the output yet in a project, but it did work for you.

The glyphs are very tightly packed. That should be OK, but I want to check as it could easily uncover a (real) edge case,

There's no option to have the tool choose an optimal texture size. For compressed fonts in the Moddable SDK that doesn't matter since the bitmap isn't deployed. That's the usual case. For uncompressed fonts it means a little bit of manual work to find optimal dimensions.

[wilberforce]

Perhaps we could add a flag that clips to the bounding box so that the png is as small as it can be. Worst case run through another external tool.

[vladimirgamalyan]

Hi guys!
I eventually added some new options to fontbm which may be useful for you:
--texture-name-suffix none to remove paging number from filenames
--texture-crop-width and --texture-crop-height to crop output textures
--texture-size to set allowed texture size list

I don't know if this is needed yet, but thanks anyway for the ideas!

[phoddie]

@vladimirgamalyan - That's great. Thank you! fontbm works very nicely. We'd like to integrate that support into the Moddable SDK. It would give developers an easy way to incorporate any font into their projects and eliminate the need o purchase a dedicated tool like Glyph Designer (with no offense at all to Glyph Designer, which has long been a solid solution for our commercial clients).

[phoddie]

@vladimirgamalyan - I tried out the latest. It works nicely across several fonts and sizes. The Moddable SDK successfully converts all the fonts output by fontbm. Even with the glyphs packed as highly as possible (no space between them in many cases) there were no visible artifacts.

There is a problem that appears to be with metrics. I haven't had a chance to investigate the source of the problem. I'm documenting it here for now. Maybe you will have some suggestion. To test I used the Commodetto text example lightly modified to use the font files output by fontbm. The conversion command line used is:

./fontbm --font-file ./tests/fonts/FreeSans.ttf --font-size 48 --output $MODDABLE/examples/commodetto/text/test --texture-crop-height --texture-name-suffix none --data-format bin

Here's the output:

Notice that "Right" is cropped on the "t" but it should be completely visible, stopping 2 pixels from the right edge. In the run "Clip text tha" the baselines appear off (see the adjacent "t" and "h"). Maybe that is just the vertical anti-aliasing, but it doesn't seem quite right.

In a test with a bold italic variant of Segoe, the font height was off somehow, so that the "p" in "Clip" was clipped off the bottom of the screen.

[vladimirgamalyan]

@phoddie It looks like the t glyph from FreeSans.ttf is lower than the others, this is how it looks in FontForge:

However, Windows rendering engine renders the glyph about 1px higher (left) then FreeType (right):

I'll try to dig into the difference (perhaps both engines do it as they thinks is correct, but there might also be a rounding issue somewhere in fontbm).

Аbout right-aligning text, I guess there is some difference between text rendering and text-width calculation procedures. Usually, for right-aligned text at first you need to calculate total text width (which is the final cursor position after print all characters) and then print at target coordinates minus the width. Accordingly, if you have different cursor increments when calculating and printing, then the right side does not match.

And about "Clip" was clipped off the bottom of the screen. Probably, you use the common/lineHeight value as total text line height, but it is only distance between two lines of text, and some glyphs can go beyond the boundaries of their line.

[phoddie]

@vladimirgamalyan, thank you for the notes and guidance!

We don't generally have clipping (horizontal or vertical)) problems with the fonts we generate from Glyph Designer. But, the output of fontbm may exercise different features of the format and so reveal bugs in our code. I'll investigate based on your feedback.

FWIW – the comparison of rendering between FreeType and Windows is interesting. There are differences across renderers based on priorities, so it is possibly just that. I personally prefer sharp edges where possible, like Windows does here. I did some work on generating sharper TrueType glyphs a while ago. But, it may well be that FreeType's output is a more accurate representation of the glyph.

[phoddie]

@vladimirgamalyan - the right-edge horizontal clipping problem is my mistake. I was experimenting with local changes. Removing those gives the expected result. Apologies for the noise.

The vertical clipping problem is more curious. Maybe you have an idea? I used this Segoe font. The conversion command line is:

./fontbm --font-file ~/Desktop/segoe.ttf --font-size 40 --output $MODDABLE/examples/commodetto/text/test --texture-crop-height --texture-name-suffix none --data-format bin

I modified the test app to display the bounding rectangle of the run. The y coordinate for the drawing is the top of the color band.

The descender is outside the bounds, as you noted it can be sometimes. But, the problem appears to be that there is an unexpectedly large amount of space above the glyph. This is due to the yoffset field in each char, which range from 22 to 31. The base is 52 and lineHeight is 54, which leaves just 2 pixels for the descenders. We ignore scaleW and scaleW but they appear to both be 1.0 (256) so that doesn't appear to be a problem here.

[vladimirgamalyan]

Well, there is one questionable decision of mine, maybe you can get me right direction, I'll try to explain. The large space above the glyphs is required to fit all possible glyphs (not even used) from the font, which gives always positive or zero yoffset and fixed base no matter what glyphs you decide to add in the future.

AngelCode's BMFont goes another way, it uses Windows TEXTMETRIC.tmAscent as base value (without TEXTMETRIC.tmInternalLeading) and crops top of glyphs above this value:

As result of this cropping - you get smaller amount of space above the glyphs and possibility to fit your text in lineHeight.

In fontbm case, lineHeight is used only for vertical shift between lines. To get required vertical space you need to calculate maximum yoffset+height from all used glyphs (for example, on parsing stage). For convenience, there is an --extra-info flag to get ready to use totalHeight value (placed in common section, for text formats only).

[phoddie]

Just like descenders can contain pixels below the line bounds, the same can be true for some glyphs above the line bounds. This often happens with accented characters. (FWIW – I recall some font engines even have an option to vertically squash glyphs to avoid that.)

As a test, I ran the same font through Glyph Designer. Here's the font it output. For convenience, it includes the Latin-1 Supplement. The base is 43 and lineHeight is 53. The lineHeight is about the same, but the base is very different, which is consistent with your notes. This is how it renders.

To get required vertical space you need to calculate maximum yoffset+height from all used glyphs (for example, on parsing stage).

This hasn't been necessary for us to-date. While it could be done, it would slow loading of fonts a bit and, I fear. require different code depending on which tool generated the font.

[vladimirgamalyan]

@phoddie I agree, it would be better to have similar values from different tools. If I understand you correctly, the base value should be the same as the font's ascent value (and some glyphs may have negative yoffset value), right? (this is how Angel's BMFont and probably Glyph Designer work)
Аlso, what do you think is the best way to set lineHeight value - as the height from font metrics (distance between two consecutive baselines) or as Ascent+Descent sum. The first method is used in fontbm, and the second in Angel's BMFont (not sure how it works in Glyph Designer).

[phoddie]

@vladimirgamalyan – I think you are correct that the issue is with the way base is determined. For my own understanding, I prepared a side-by-side test. Here's the test image I generated:

• The first line is drawn with the font from Glyph Designer.
• The second line is drawn with the font from fontbm.
• The third line draws both -- first Glyph Designer and then fontbm. To have them align, the fontbm is drawn with a dx of +1 and dy of -7.

In all cases, the white background is the base and the gray background is lineHeight - base.

Both tools generate reasonable glyphs. Their horizontal metrics are quite similar. The heights are similar but the vertical offsets differ significantly.

The lineHeight is nearly identical for both (53 and 54) so the most significant difference is base (43 and 52). The definitions from the documentation says:

• base - The number of pixels from the absolute top of the line to the base of the characters
• lineHeight - This is the distance in pixels between each line of text

If I understand you correctly, the base value should be the same as the font's ascent value (and some glyphs may have negative yoffset value), right?

The xoffset and yoffset values are signed, so it seems quite possible that they could be negative. Here are the traces of those values for the run used in the test image. Note that xoffset goes negative but not yoffset:

Glyph Designer
xoffset -3, yoffset  7
xoffset -3, yoffset  9
xoffset -3, yoffset  7
xoffset  1, yoffset 15
xoffset -2, yoffset 23
xoffset  1, yoffset 15
xoffset  2, yoffset  7

fontbm
xoffset -2, yoffset 14
xoffset -2, yoffset 16
xoffset -2, yoffset 14
xoffset  2, yoffset 23
xoffset -1, yoffset 30
xoffset  2, yoffset 23
xoffset  2, yoffset 15

AngelCode's BMFont goes another way, it uses Windows TEXTMETRIC.tmAscent as base value (without TEXTMETRIC.tmInternalLeading) and crops top of glyphs above this value:

The behavior of AngelCode's BMFont seems similar to Glyph Designer. The tmInternalLeading value seems to be intended to be included as part of tmAscent ("The amount of leading (space) inside the bounds set by the tmHeight member") and so should not modify the lineHeight as tmLineHeight is defined as the sum of tmAscent and tmDescent.

Аlso, what do you think is the best way to set lineHeight value - as the height from font metrics (distance between two consecutive baselines) or as Ascent+Descent sum. The first method is used in fontbm, and the second in Angel's BMFont (not sure how it works in Glyph Designer).

I believe the distance between two baselines is reasonable. When rendering, that is the vertical step between lines. The sum of ascent and descent gives the same result when tmExternalLeading is 0 (which may not be rare?).

[vladimirgamalyan]

@phoddie Wow, thanks for the cool research! Thank you!
I'll try to come back to this by the end of the week (unfortunately too busy right now).

[vladimirgamalyan]

Added these (base is now equal to the font ascender value) changes to the last release, the output is (red - fontbm, black - Glyph Designer, blue - base, green - lineHeight - base):

Now it looks more similar, but there is still 1-2 pixels difference. Maybe it has something to do with padding, this can be checked by setting the padding to zero in Glyph Designer.

Also I compared the difference with Hiero outputs, looks the same (red - fontbm, white - Hiero):

---

The lineHeight value still equals the distance between two baselines, but with rounding instead ceiling to integer (as noted in FreeType docs).

[phoddie]

@vladimirgamalyan – very nice! Thank you. I see similar results to yours:

The dx is still 1 and the dy is now just 2. I don't believe Glyph Designer uses FreeType, but Apple's macOS font rendering. Some variation is to be expected. The dy seems to be because of a difference in the baselines. The fontbm/FreeType output is exactly on the baseline, but the Glyph Designer/Apple output rests a couple pixels below. The difference is most noticeable on the 3/4 glyph, where the fontbm/FreeType version actually sits a half pixel above the baseline. The baseline differences may be because Apple is applying tables in the font that FreeType ignores.

In any case, the fontbm output looks very usable. And your glyph bitmap is about 20% smaller in this example, which is a meaningful gain for embedded projects.

I want to explore integrating fontbm support into our project manifest, so that developers can have fontbm conversion as part of their build rather than a separate manual step.

[wilberforce]

Great news Cheers Rhys

…

On 25/12/2021, at 7:11 AM, Peter Hoddie ***@***.***> wrote:  @vladimirgamalyan – very nice! Thank you. I see similar results to yours: The dx is still 1 and the dy is now just 2. I don't believe Glyph Designer uses FreeType, but Apple's macOS font rendering. Some variation is to be expected. The dy seems to be because of a difference in the baselines. The fontbm/FreeType output is exactly on the baseline, but the Glyph Designer/Apple output rests a couple pixels below. The difference is most noticeable on the 3/4 glyph, where the fontbm/FreeType actually sits a half pixels above the baseline. The baseline differences may be because Apple is applying tables in the font that FreeType ignores. In any case, the fontbm output looks very usable. And your glyph bitmap is about 20% smaller in this example, which is a meaningful gain for embedded projects. I want to explore integrating fontbm support into our project manifest, so that developers can have fontbm conversion as part of their build rather than a separate manual step. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[phoddie]

@vladimirgamalyan – I've been working on integrating fontbm with out build tools in spare moments. It is going nicely. When I enabled kerning, I got some unexpected results. I'll summarize those in an issue in your repository.

[phoddie]

The Moddable SDK now contains support for fontbm! More fonts. Smaller fonts. More languages. All good.

I wrote it about it in the blog post, "Using More Fonts More Easily in IoT Products". The documentation on building fonts has been updated with the technical details.

Many thanks to @vladimirgamalyan for fontbm and his outstanding support!

After nearly two years, time to close out this issue.