Entry fails to save when embed field contains an instagram image with emojis in the caption

[dohayon]

[leevigraham]

Interesting… I'll check with @brandonkelly how craft handles emojis

[brandonkelly]

@leevigraham Craft emoji support has 3 facets:

• craft\helpers\StringHelper::encodeMb4() will HTML-encode all 4(+)-byte characters. Only con is that they get converted to HTML entities, which may not be desired if the string isn’t HTML.

• LitEmoji\LitEmoji::unicodeToShortcode() will convert most emoji to :shortcodes; and shortcodeToUnicode() will convert :shortcodes: back to the unicode emjoi chars. Only con is that any unsupported emoji characters will not be touched, which could still lead to errors.

• PostgreSQL has no trouble with emoji, so if you have a site that needs good emoji support, you should use that if at all possible instead of MySQL.

[leevigraham]

Sounds like craft\helpers\StringHelper::encodeMb4() is the best option here given the output will always be html.

[leevigraham]

Example URL: https://www.instagram.com/p/BfYPK64FFe6/?taken-by=leevigraham

[leevigraham]

Looking at the craft code it looks like fields are using encodeMb4() in the fckeditor plugin

if (Craft::$app->getDb()->getIsMysql()) {
    // Encode any 4-byte UTF-8 characters.
    $value = StringHelper::encodeMb4($value);
}

The EmbedModel implements JsonSerializable which means it's skipped by Crafts serializeValue method…

Interestingly if you try to submit an emoji in a page title then the same issue occurs:

SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x98\x8Afu...' for column 'title' at row 1
The SQL being executed was: INSERT INTO `content` (`elementId`, `siteId`, `title`, `field_gender`, `field_address`, `field_email`, `field_embed`, `field_personName`, `field_telephone`, `field_telephone2`, `dateCreated`, `dateUpdated`, `uid`) VALUES (89, 1, ' 😊fun', NULL, NULL, NULL, NULL, NULL, NULL, NULL, '2018-03-01 10:27:50', '2018-03-01 10:27:50', '60cbea1b-8b6b-430f-869c-64cc95e8fd4f')

[leevigraham]

@brandonkelly what is Craft's position on emoji support? If support is only guaranteed on a DB level then I'm tempted to follow suit.

[leevigraham]

See:

• Add support for emoji 🙏 craftcms/cms#1627
• Emoji makes plain text fields implode craftcms/cms#1753
• Emoji in entry titles throw a DB errors craftcms/cms#2513

[leevigraham]

@dohayon, I've implemented LitEmoji encoding / decoding. Let me know how the latest version goes

[leevigraham]

Related: craftcms/cms#2527