This repository has been archived by the owner on Aug 8, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
[ios] POC: implementation of a CoreText-based LocalGlyphRasterizer #10572
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
6a1a5ee
POC implementation of a CoreText-based LocalGlyphRasterizer.
ChrisLoer c43dd57
Fix build for iOS.
ChrisLoer 629565f
Adjust drawing positions and glyph metrics to fit entire glyph within…
ChrisLoer ae6dd46
Reduce padding on CJK glyphs (should make rendering slightly faster)
ChrisLoer 8e55428
Moving towards configurable darwin implementation of LocalGlyphRaster…
ChrisLoer e5c4d96
Share CFHandle code between image.mm and local_glyph_rasterizer.mm
ChrisLoer 3114947
LocalGlyphRasterizer takes a void* configuration input. If nothing is…
ChrisLoer bfefc93
Change LocalGlyphRasterizer configuration to just a plain "font famil…
ChrisLoer 3d7669b
Fix unit tests.
ChrisLoer 8a9009c
Fix non ios/macos builds.
ChrisLoer 6451d4a
Document CFHandle
ChrisLoer bf53bf4
Load fonts per-fontstack (but still no heuristics for choosing font w…
ChrisLoer 0845907
Implement font stack-based heuristics for font loading.
ChrisLoer 34e7ae2
Remove entry point for setting local ideographic font family, so that…
ChrisLoer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
#pragma once | ||
|
||
/* | ||
CFHandle is a minimal wrapper designed to hold and release CoreFoundation-style handles | ||
It is non-transferrable: wrap it in something like a unique_ptr if you need to pass it around, | ||
or just use unique_ptr with a custom deleter. | ||
CFHandle has no special treatment for null handles -- be careful not to let it hold a null | ||
handle if the behavior of the Releaser isn't defined for null. | ||
|
||
ex: | ||
using CFDataHandle = CFHandle<CFDataRef, CFTypeRef, CFRelease>; | ||
|
||
CFDataHandle data(CFDataCreateWithBytesNoCopy( | ||
kCFAllocatorDefault, reinterpret_cast<const unsigned char*>(source.data()), source.size(), | ||
kCFAllocatorNull)); | ||
*/ | ||
|
||
namespace { | ||
|
||
template <typename HandleType, typename ReleaserArgumentType, void (*Releaser)(ReleaserArgumentType)> | ||
struct CFHandle { | ||
CFHandle(HandleType handle_): handle(handle_) {} | ||
~CFHandle() { Releaser(handle); } | ||
HandleType operator*() { return handle; } | ||
operator bool() { return handle; } | ||
private: | ||
HandleType handle; | ||
}; | ||
|
||
} // namespace | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,233 @@ | ||
#include <mbgl/text/local_glyph_rasterizer.hpp> | ||
#include <mbgl/util/i18n.hpp> | ||
#include <mbgl/util/platform.hpp> | ||
|
||
#include <unordered_map> | ||
|
||
#import <Foundation/Foundation.h> | ||
#import <CoreText/CoreText.h> | ||
#import <ImageIO/ImageIO.h> | ||
|
||
#import "CFHandle.hpp" | ||
|
||
namespace mbgl { | ||
|
||
/* | ||
Darwin implementation of LocalGlyphRasterizer: | ||
Draws CJK glyphs using locally available fonts. | ||
|
||
Mirrors GL JS implementation in that: | ||
- Only CJK glyphs are drawn locally (because we can guess their metrics effectively) | ||
* Render size/metrics determined experimentally by rendering a few different fonts | ||
- Configuration is done at map creation time by setting a "font family" | ||
* JS uses a CSS font-family, this uses kCTFontFamilyNameAttribute which has | ||
somewhat different behavior. | ||
- We use heuristics to extract a font-weight based on the incoming font stack | ||
|
||
Further improvements are possible: | ||
- If we could reliably extract glyph metrics, we wouldn't be limited to CJK glyphs | ||
- We could push the font configuration down to individual style layers, which would | ||
allow any current style to be reproducible using local fonts. | ||
- Instead of just exposing "font family" as a configuration, we could expose a richer | ||
CTFontDescriptor configuration option (although we'd have to override font size to | ||
make sure it stayed at 24pt). | ||
- Because Apple exposes glyph paths via `CTFontCreatePathForGlyph` we could potentially | ||
render directly to SDF instead of going through TinySDF -- although it's not clear | ||
how much of an improvement it would be. | ||
*/ | ||
|
||
using CGColorSpaceHandle = CFHandle<CGColorSpaceRef, CGColorSpaceRef, CGColorSpaceRelease>; | ||
using CGContextHandle = CFHandle<CGContextRef, CGContextRef, CGContextRelease>; | ||
using CFStringRefHandle = CFHandle<CFStringRef, CFTypeRef, CFRelease>; | ||
using CFAttributedStringRefHandle = CFHandle<CFAttributedStringRef, CFTypeRef, CFRelease>; | ||
using CFDictionaryRefHandle = CFHandle<CFDictionaryRef, CFTypeRef, CFRelease>; | ||
using CTFontDescriptorRefHandle = CFHandle<CTFontDescriptorRef, CFTypeRef, CFRelease>; | ||
using CTLineRefHandle = CFHandle<CTLineRef, CFTypeRef, CFRelease>; | ||
|
||
class LocalGlyphRasterizer::Impl { | ||
public: | ||
Impl(const optional<std::string> fontFamily_) | ||
: fontFamily(fontFamily_) | ||
{} | ||
|
||
~Impl() { | ||
for (auto& pair : fontHandles) { | ||
CFRelease(pair.second); | ||
} | ||
} | ||
|
||
|
||
CTFontRef getFont(const FontStack& fontStack) { | ||
if (!fontFamily) { | ||
return NULL; | ||
} | ||
|
||
if (fontHandles.find(fontStack) == fontHandles.end()) { | ||
|
||
NSDictionary* fontTraits = @{ (NSString *)kCTFontSymbolicTrait: [NSNumber numberWithFloat:getFontWeight(fontStack)] }; | ||
|
||
NSDictionary *fontAttributes = @{ | ||
(NSString *)kCTFontSizeAttribute: [NSNumber numberWithFloat:24.0], | ||
(NSString *)kCTFontFamilyNameAttribute: [[NSString alloc] initWithCString:fontFamily->c_str() encoding:NSUTF8StringEncoding], | ||
(NSString *)kCTFontTraitsAttribute: fontTraits | ||
//(NSString *)kCTFontStyleNameAttribute: (getFontWeight(fontStack) > .3) ? @"Bold" : @"Regular" | ||
}; | ||
|
||
CTFontDescriptorRefHandle descriptor(CTFontDescriptorCreateWithAttributes((CFDictionaryRef)fontAttributes)); | ||
CTFontRef font = CTFontCreateWithFontDescriptor(*descriptor, 0.0, NULL); | ||
if (!font) { | ||
throw std::runtime_error("CTFontCreateWithFontDescriptor failed"); | ||
} | ||
|
||
fontHandles[fontStack] = font; | ||
} | ||
return fontHandles[fontStack]; | ||
} | ||
|
||
private: | ||
float getFontWeight(const FontStack& fontStack) { | ||
// Analog to logic in glyph_manager.js | ||
// From NSFontDescriptor.h (macOS 10.11+) NSFontWeight*: | ||
constexpr float light = -.4; | ||
constexpr float regular = 0.0; | ||
constexpr float medium = .23; | ||
constexpr float bold = .4; | ||
|
||
float fontWeight = regular; | ||
for (auto font : fontStack) { | ||
// Last font in the fontstack "wins" | ||
std::string lowercaseFont = mbgl::platform::lowercase(font); | ||
if (lowercaseFont.find("bold") != std::string::npos) { | ||
fontWeight = bold; | ||
} else if (lowercaseFont.find("medium") != std::string::npos) { | ||
fontWeight = medium; | ||
} else if (lowercaseFont.find("light") != std::string::npos) { | ||
fontWeight = light; | ||
} | ||
} | ||
|
||
return fontWeight; | ||
} | ||
|
||
std::unordered_map<FontStack, CTFontRef, FontStackHash> fontHandles; | ||
optional<std::string> fontFamily; | ||
}; | ||
|
||
LocalGlyphRasterizer::LocalGlyphRasterizer(const optional<std::string> fontFamily) | ||
: impl(std::make_unique<Impl>(fontFamily)) | ||
{} | ||
|
||
LocalGlyphRasterizer::~LocalGlyphRasterizer() | ||
{} | ||
|
||
bool LocalGlyphRasterizer::canRasterizeGlyph(const FontStack& fontStack, GlyphID glyphID) { | ||
return util::i18n::allowsFixedWidthGlyphGeneration(glyphID) && impl->getFont(fontStack); | ||
} | ||
|
||
/* | ||
// TODO: In theory we should be able to transform user-coordinate bounding box and advance | ||
// values into pixel glyph metrics. This would remove the need to use fixed glyph metrics | ||
// (which will be slightly off depending on the font), and allow us to return non CJK glyphs | ||
// (which will have variable "advance" values). | ||
void extractGlyphMetrics(CTFontRef font, CTLineRef line) { | ||
CFArrayRef glyphRuns = CTLineGetGlyphRuns(line); | ||
CFIndex runCount = CFArrayGetCount(glyphRuns); | ||
assert(runCount == 1); | ||
CTRunRef run = (CTRunRef)CFArrayGetValueAtIndex(glyphRuns, 0); | ||
CFIndex glyphCount = CTRunGetGlyphCount(run); | ||
assert(glyphCount == 1); | ||
const CGGlyph *glyphs = CTRunGetGlyphsPtr(run); | ||
|
||
CGRect boundingRects[1]; | ||
boundingRects[0] = CGRectMake(0, 0, 0, 0); | ||
CGSize advances[1]; | ||
advances[0] = CGSizeMake(0,0); | ||
CGRect totalBoundingRect = CTFontGetBoundingRectsForGlyphs(font, kCTFontOrientationDefault, glyphs, boundingRects, 1); | ||
double totalAdvance = CTFontGetAdvancesForGlyphs(font, kCTFontOrientationDefault, glyphs, advances, 1); | ||
|
||
// Break in the debugger to see these values: translating from "user coordinates" to bitmap pixel coordinates | ||
// should be OK, but a lot of glyphs seem to have empty bounding boxes...? | ||
(void)totalBoundingRect; | ||
(void)totalAdvance; | ||
} | ||
*/ | ||
|
||
PremultipliedImage drawGlyphBitmap(GlyphID glyphID, CTFontRef font, Size size) { | ||
PremultipliedImage rgbaBitmap(size); | ||
|
||
CFStringRefHandle string(CFStringCreateWithCharacters(NULL, reinterpret_cast<UniChar*>(&glyphID), 1)); | ||
|
||
CGColorSpaceHandle colorSpace(CGColorSpaceCreateDeviceRGB()); | ||
// TODO: Is there a way to just draw a single alpha channel instead of copying it out of an RGB image? Doesn't seem like the grayscale colorspace is what I'm looking for... | ||
if (!colorSpace) { | ||
throw std::runtime_error("CGColorSpaceCreateDeviceRGB failed"); | ||
} | ||
|
||
constexpr const size_t bitsPerComponent = 8; | ||
constexpr const size_t bytesPerPixel = 4; | ||
const size_t bytesPerRow = bytesPerPixel * size.width; | ||
|
||
CGContextHandle context(CGBitmapContextCreate( | ||
rgbaBitmap.data.get(), | ||
size.width, | ||
size.height, | ||
bitsPerComponent, | ||
bytesPerRow, | ||
*colorSpace, | ||
kCGBitmapByteOrderDefault | kCGImageAlphaPremultipliedLast)); | ||
if (!context) { | ||
throw std::runtime_error("CGBitmapContextCreate failed"); | ||
} | ||
|
||
CFStringRef keys[] = { kCTFontAttributeName }; | ||
CFTypeRef values[] = { font }; | ||
|
||
CFDictionaryRefHandle attributes( | ||
CFDictionaryCreate(kCFAllocatorDefault, (const void**)&keys, | ||
(const void**)&values, sizeof(keys) / sizeof(keys[0]), | ||
&kCFTypeDictionaryKeyCallBacks, | ||
&kCFTypeDictionaryValueCallBacks)); | ||
|
||
CFAttributedStringRefHandle attrString(CFAttributedStringCreate(kCFAllocatorDefault, *string, *attributes)); | ||
|
||
CTLineRefHandle line(CTLineCreateWithAttributedString(*attrString)); | ||
|
||
// For debugging only, doesn't get useful metrics yet | ||
//extractGlyphMetrics(font, *line); | ||
|
||
// Start drawing a little bit below the top of the bitmap | ||
CGContextSetTextPosition(*context, 0.0, 5.0); | ||
CTLineDraw(*line, *context); | ||
|
||
return rgbaBitmap; | ||
} | ||
|
||
Glyph LocalGlyphRasterizer::rasterizeGlyph(const FontStack& fontStack, GlyphID glyphID) { | ||
Glyph fixedMetrics; | ||
CTFontRef font = impl->getFont(fontStack); | ||
if (!font) { | ||
return fixedMetrics; | ||
} | ||
|
||
fixedMetrics.id = glyphID; | ||
|
||
Size size(35, 35); | ||
|
||
fixedMetrics.metrics.width = size.width; | ||
fixedMetrics.metrics.height = size.height; | ||
fixedMetrics.metrics.left = 3; | ||
fixedMetrics.metrics.top = -1; | ||
fixedMetrics.metrics.advance = 24; | ||
|
||
PremultipliedImage rgbaBitmap = drawGlyphBitmap(glyphID, font, size); | ||
|
||
// Copy alpha values from RGBA bitmap into the AlphaImage output | ||
fixedMetrics.bitmap = AlphaImage(size); | ||
for (uint32_t i = 0; i < size.width * size.height; i++) { | ||
fixedMetrics.bitmap.data[i] = rgbaBitmap.data[4 * i + 3]; | ||
} | ||
|
||
return fixedMetrics; | ||
} | ||
|
||
} // namespace mbgl |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing
#pragma once
We also typically make sure that header files include all of the symbols they reference. Instead of defining all of the aliases in the header file, we can continue to define them where we need them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
Maybe we don't need CFHandle at all? It seems like something we could replace with a portable "unique handle holder" or maybe just
std::unique_ptr
with custom deleters?