URL previews includes unicode punycode which causes issues URLs in the text body

[matrixbot]

This issue has been migrated from #14068.

---

Description

element-hq/element-web#23432

Synapse injects “ (5o0a) punycode into og:description of twitter URLs, leading to broken links.

Steps to reproduce

Forwarded Element issue:

Steps to reproduce

1. Post a twitter URL into a room, URL previews enabled
2. Twitter post content is put into double quotes ""
3. in case of media-only posts, the t.co media URL is in quotes
4. element includes the ending quote in the URL, resulting in a broken link

Outcome

What did you expect?

links work

What happened instead?

Demo URLs for reference:
https://twitter.com/FXNetworks/status/1577704289476128771
https://twitter.com/mischiefanimals/status/1576904037449969664

Operating system

arch

Application version

Element Nightly version: 2022100501 Olm version: 3.2.12

How did you install the app?

aur

Homeserver

private

Synapse Version

1.68.0

Installation Method

Docker (matrixdotorg/synapse)

Platform

debian, matrix-docker-ansible-deploy

Relevant log output

n/a

Anything else that would be useful to know?

curl "https://publish.twitter.com/oembed?url=https://twitter.com/mischiefanimals/status/1576904037449969664" | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   853  100   853    0     0   3946      0 --:--:-- --:--:-- --:--:--  4140
{
  "url": "https://twitter.com/mischiefanimals/status/1576904037449969664",
  "author_name": "animals going goblin mode",
  "author_url": "https://twitter.com/mischiefanimals",
  "html": "<blockquote class=\"twitter-tweet\"><p lang=\"zxx\" dir=\"ltr\"><a href=\"https://t.co/fVP8YWHS2j\">pic.twitter.com/fVP8YWHS2j</a></p>&mdash; animals going goblin mode (@mischiefanimals) <a href=\"https://twitter.com/mischiefanimals/status/1576904037449969664?ref_src=twsrc%5Etfw\">October 3, 2022</a></blockquote>\n<script async src=\"https://platform.twitter.com/widgets.js\" charset=\"utf-8\"></script>\n",
  "width": 550,
  "height": null,
  "type": "rich",
  "cache_age": "3153600000",
  "provider_name": "Twitter",
  "provider_url": "https://twitter.com",
  "version": "1.0"
}

https://user-images.githubusercontent.com/2403652/194146661-044758f9-fefd-4744-9ef2-bd3aec094d40.png