Fix images in the RSS feed #1245

rogermparent · 2020-05-05T20:34:48Z

It seems many Gatsby sites don't really have images within their RSS feeds' content, and this turned out to be slightly more involved than I thought it would.

Basically, this branch changes the way the feed content is parsed. Instead of taking pre-rendered HTML from GraphQL, it uses the Rehype htmlAst that gatsby-transformer-remark already makes for us and transforms links and images into a simpler form of HTML that works with RSS.

To get images working, I had to do two things to the feed's AST before rendering to HTML:

Change root-relative src attributes (/example) to absolute links (https://dvc.org/example)
"Unwrap" responsive image wrappers by replacing them with the img tag fallback within.

Fixes the images part of #1177, and since titles are already fixed it fixes #1177 entirely.

This runs the feed's htmlAst (from the remark transformer) through a processor that changes all root-relative links (/example) into absolute links (https://dvc.org/example), which is what many non-browser consumers of HTML expect.

This commit combines the root-relative rewriting with now-working image unwrapping to deliver working images to the RSS feed! The processor definition is also moved into the feed scope to take advantage of the feed query to get the root URL.

Since moving it out of gatsby-config won't be as simple as I thought, I decided to leave it out of the scope of this branch.

shcheklein · 2020-05-05T20:49:52Z

Looks too complicated and affects the website (optimization for smaller screen sizes, etc?). Also, two notes:

I see that other images are rendered in RSS even in the current version.
We should be applying these changes to the RSS render, w/o affecting and complicating even further the website renderer.

shcheklein · 2020-05-05T20:50:28Z

If we can't do these two points ^^ - let's just ignore RSS issue for now - it's not the biggest priority.

rogermparent · 2020-05-05T21:07:31Z

This change is only on the RSS render, you can see the site's images are still totally responsive on the deploy preview. These changes all live in gatsby-plugin-feed's config, and do not touch the site render.

shcheklein · 2020-05-05T22:06:15Z

Oh, sorry, @rogermparent - function is already big enough that I missed the context.

How is it possible though that other images do work in RSS w/o applying this transformation?

can it be somehow related to some custom image processing plugins/logic that we already have?

rogermparent · 2020-05-05T22:26:56Z

Oh, sorry, @rogermparent - function is already big enough that I missed the context.

How is it possible though that other images do work in RSS w/o applying this transformation?

can it be somehow related to some custom image processing plugins/logic that we already have?

No worries! I'm aware gatsby-config is getting packed, but there are some "global" variables that need some refactoring before we can move things into their own files, and I decided it was out of the scope of this branch.

I'm not sure which images you're talking about that do work, as none of the images in the body work for me. Embeds work, but those are just iframes. Any images that aren't wrapped with a gatsby-image component, like external ones, also should work currently although I haven't seen them.

The problem is twofold- the HTML from the remark plugin has gatsby-image responsive images, which are actually a bunch of different elements wrapped together, embedded in its HTML. This works fantastically for real web browsers, but for more simple embedded browsers like RSS readers the trick actually breaks things.

The other half is root-relative links, which are a browser-specific feature that falls apart outside of that context. Once you have some way to reliably alter HTML properties (rehype, in this case, as it's the highly compatible sister project to remark), it's incredibly easy to make all root-relative links into absolute by simply prepending the site URL to them, much like how gatsby-plugin-feed uses links outside of the content and requires the same site_url variable to do so.

shcheklein · 2020-05-05T22:30:40Z

I'm not sure which images you're talking about that do work

hmm : 🤔 This is the most recent post in feedly for me -

rogermparent · 2020-05-05T22:43:34Z

I just looked at that image, and it's broken for me on a linux-based local reader called FeedReader.
That image in particular is an SVG, which can't be processed by Sharp (nor would it make sense to) and doesn't become a Gatsby Image like raster formats.

This means those particular SVG images bypass the Gatsby-image problem but still suffer from the root-relative problem. The latter must be an issue that only certain readers suffer from.

gatsby-config.js

yarn.lock

gatsby-config.js

Also changed a bit around implementation: 1. Replaced `rehype-urls`, which depended on older rehype libs, with a local implementation of the same thing for root-to-absolute linking. 2. The new feed util leans on the existing rehype-to-html function, and now the new processor `.use`s less `unified` plugins as it only does AST processing.

rogermparent

Extracted the function out and fixed many issues, but could use a couple final tweaks.

gatsby-config.js

yarn.lock

gatsby-config.js

shcheklein · 2020-05-06T21:20:22Z

plugins/utils/makeFeedHtml.js

+const { select, selectAll } = require('hast-util-select')
+
+const rootToAbsolute = siteUrl => tree => {
+  selectAll('a', tree).forEach(node => {


let's put a link here gatsbyjs/gatsby#14133 to remove this when it's fixed upstream?

Sounds good to me!

shcheklein · 2020-05-06T21:23:47Z

plugins/utils/makeFeedHtml.js

+  return tree
+}
+
+function makeFeedHtml(htmlAst, siteUrl) {


really annoying that gastby plugin feed does not handle this ... kinda feels like all of this is a requirement for that plugin to be useful

I agree to a certain extent, especially for our use case, but RSS feeds are very often used for something like podcasts where the encoded content is more of a summary than the whole blog (ads are a lot harder in an RSS reader, after all), and the encoded content is kept simple.

This isn't to say the default and suggested queries of gatsby-plugin-feed are good, I think naively passing through the content's raw html string is kind of poor way to go about things, but it's not as if the plugin is useless without this kind of content reprocessing.

shcheklein · 2020-05-06T21:24:42Z

plugins/utils/makeFeedHtml.js

+        ...fallbackImg,
+        properties: {
+          ...fallbackImg.properties,
+          style: 'max-width: 100%; margin: auto;'


qq- does it makes/do we respect our custom image sizing logic?

This just changes the wrapper's internal image to have max-width: 100% so larger images don't extend offscreen on narrower displays and margin: auto so smaller ones are centered.
The original max-width on the wrapper that the existing plugin operates on is untouched, as well as any other properties on the wrapper.

rogermparent added 4 commits May 4, 2020 20:53

Make root-relative links absolute in feed.

3ec0c30

This runs the feed's htmlAst (from the remark transformer) through a processor that changes all root-relative links (/example) into absolute links (https://dvc.org/example), which is what many non-browser consumers of HTML expect.

Merge branch 'master' into feed-images

1af8716

Move the feed plugin back to its original space.

20943ff

Since moving it out of gatsby-config won't be as simple as I thought, I decided to leave it out of the scope of this branch.

shcheklein temporarily deployed to dvc-landing-feed-images-gm9bcy May 5, 2020 20:35 Inactive

shcheklein reviewed May 6, 2020

View reviewed changes

gatsby-config.js Outdated Show resolved Hide resolved

shcheklein reviewed May 6, 2020

View reviewed changes

gatsby-config.js Outdated Show resolved Hide resolved

shcheklein reviewed May 6, 2020

View reviewed changes

yarn.lock Outdated Show resolved Hide resolved

shcheklein reviewed May 6, 2020

View reviewed changes

gatsby-config.js Outdated Show resolved Hide resolved

shcheklein reviewed May 6, 2020

View reviewed changes

gatsby-config.js Outdated Show resolved Hide resolved

rogermparent added 5 commits May 5, 2020 22:04

Merge branch 'master' into feed-images

d2ba7eb

Change unwrapped image styling in a way that respects original max-width

7fd9e8b

Revert package.json to its pre-branch form.

a10217c

Merge current master into feed-images

dbae74c

rogermparent temporarily deployed to dvc-landing-feed-images-gm9bcy May 6, 2020 20:08 Inactive

rogermparent commented May 6, 2020

View reviewed changes

gatsby-config.js Outdated Show resolved Hide resolved

gatsby-config.js Outdated Show resolved Hide resolved

yarn.lock Outdated Show resolved Hide resolved

rogermparent added 2 commits May 6, 2020 16:32

Use remark image wrapper class from its exported constants

d9ecb1f

Clean formatting of blog feed query

fd1dcf9

rogermparent temporarily deployed to dvc-landing-feed-images-gm9bcy May 6, 2020 20:34 Inactive

shcheklein reviewed May 6, 2020

View reviewed changes

gatsby-config.js Outdated Show resolved Hide resolved

shcheklein reviewed May 6, 2020

View reviewed changes

Change plugin definitions to all have "resolve" first.

b1bb845

shcheklein reviewed May 6, 2020

View reviewed changes

rogermparent temporarily deployed to dvc-landing-feed-images-gm9bcy May 6, 2020 21:26 Inactive

shcheklein approved these changes May 6, 2020

View reviewed changes

shcheklein merged commit 2ea63aa into master May 6, 2020

weekly-digest bot mentioned this pull request May 10, 2020

Weekly Digest (3 May, 2020 - 10 May, 2020) #1276

Closed

shcheklein deleted the feed-images branch February 28, 2021 19:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix images in the RSS feed #1245

Fix images in the RSS feed #1245

rogermparent commented May 5, 2020 •

edited

Loading

shcheklein commented May 5, 2020

shcheklein commented May 5, 2020

rogermparent commented May 5, 2020 •

edited

Loading

shcheklein commented May 5, 2020

rogermparent commented May 5, 2020 •

edited

Loading

shcheklein commented May 5, 2020

rogermparent commented May 5, 2020 •

edited

Loading

rogermparent left a comment

shcheklein May 6, 2020

rogermparent May 6, 2020

shcheklein May 6, 2020

rogermparent May 6, 2020

shcheklein May 6, 2020

rogermparent May 6, 2020

Fix images in the RSS feed #1245

Fix images in the RSS feed #1245

Conversation

rogermparent commented May 5, 2020 • edited Loading

shcheklein commented May 5, 2020

shcheklein commented May 5, 2020

rogermparent commented May 5, 2020 • edited Loading

shcheklein commented May 5, 2020

rogermparent commented May 5, 2020 • edited Loading

shcheklein commented May 5, 2020

rogermparent commented May 5, 2020 • edited Loading

rogermparent left a comment

Choose a reason for hiding this comment

shcheklein May 6, 2020

Choose a reason for hiding this comment

rogermparent May 6, 2020

Choose a reason for hiding this comment

shcheklein May 6, 2020

Choose a reason for hiding this comment

rogermparent May 6, 2020

Choose a reason for hiding this comment

shcheklein May 6, 2020

Choose a reason for hiding this comment

rogermparent May 6, 2020

Choose a reason for hiding this comment

rogermparent commented May 5, 2020 •

edited

Loading

rogermparent commented May 5, 2020 •

edited

Loading

rogermparent commented May 5, 2020 •

edited

Loading

rogermparent commented May 5, 2020 •

edited

Loading