Updating gatsby-plugin-manifest

[moonmeister]

The problem

There are currently two plugins in the Gatsby eco system that deal with the web app manifest, favicons,
and apple icons (gatsby-plugin-manifest and gatsby-plugin-favicon). The latter auto generates these images but allows no customization of the other manifest properties. The former allows complete customization while not providing auto generation of the various icons.

The goal

In short the goal of this PR is to update the existing gatsby-plugin-manifest to include auto generation of images and maintain it's great flexibility in configuring all aspects of the web app manifest.

The challenges

While I was hoping to "simply" combined the two plugins it's not going to be so easy (of course not, how naive of me). The gatsby-plugin-favicon relies on another npm module that acts as a plugin to webpack favicons-webpack-plugin. That module in turn relies on the favicons.

The biggest problem is the favicons-webpack-plugin doesn't support the web app manifest configs for anything other that name and background-color. You can't configure any other parameters. The underlying favicons plugin does support more, if not all the various web app manifest parameters, it's quite out of date. For Firefox, it's still generating what's now defunct spec. There's a couple other weir quirks I won't get into.

The solution

Theory

Now that the Chrome spec is becoming standardized by [W3](https://www.w3.org/TR/appmanifest/, this plugin should reflect that standard.

I love how the current gatsby-plugin-manifest simply passes the plugin config on to be the webapp manifest. This means our plugin will maintain compatability to the spec without modification...people simply have to update their config. Obviously, if we want to add new fields to have default values we can do that.

This will need to be augmented by some boolean values that determine what images are created and what part of the icons config is added.

webapp: true,
ie: true.
apple: true,
favicon: true,

The backwards compatibility issue

Currently only Edge maybe, Firefox, Opera, and Chrome support the web app manifest standard. We have to account for that...

What's up with the ie, apple, favicons boolean values above? This controls weather our web app includes support for these non-standard standards. IE and Safari/Apple don't support the web app manifest standard. Favicons are still a thing though (this is total speculation) may be eclipsed by the web app manifest standard. But for now we should probably keep generating them.

Reality

IMO, basically nothing from the gatsby-plugin-favicons plugin is useful moving forward. Given that gatsby-plugin-sharp exists I'm thinking that we could leverage that to generate the needed images from a single source file. The plugin would auto populate the icons attribute of the manifest.json and all other attributes in the manifest.json would be passed through from the plugin config. Any other files, headers, icons for IE, Apple, and the favicon would also be generate automatically. Any data in IE's browserconfig.xml would be derived from the web app manifest fields.

In conclusion...

So what code did you just submit?

Basically, I modified to gatsby-plugin-manifest to be gatsby-plugin-favicon and then tried to modify the code to be able to configure the manifest.json. But, due to the previously mentioned issues the only thing this code adds is the ability to modify the name and background-color fields. So, this code is functionally worthless and should never see release, it was useful in that I learned everything above...fun times.

Feedback:

Please give me feed back on any thoughts on how this plugin should function in it's v2 state. The one question I specifically have is:

• Should v2 of this plugin be backwards compatible? It'd be difficult, probably not impossible. If we're concerned about backwards compatibility a better option might be a different plugin name (this comes with it's own confusion though).

Resources

Are lots of resources and reference guides for the various specs mentioned in this PR. While this exists I don't believe it's entirely accurate... I could be wrong though.

Specs

Web app Manifest W3 Spec: https://www.w3.org/TR/appmanifest/
Favicon: https://www.w3.org/2005/10/howto-favicon

Individual Browser Reference

Chrome: https://developers.google.com/web/fundamentals/web-app-manifest/
Apple: https://developer.apple.com/library/content/documentation/AppleApplications/Reference/SafariWebContent/ConfiguringWebApplications/ConfiguringWebApplications.html
Internet Explore: https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/platform-apis/dn320426(v=vs.85)
FireFox: https://developer.mozilla.org/en-US/docs/Web/Manifest
Opera: https://dev.opera.com/articles/installable-web-apps/

[KyleAMathews]

Deploy preview for using-glamor failed.

Built with commit e7e1fede50eb112cb786fa88c92f51c5d82cb284

https://app.netlify.com/sites/using-glamor/deploys/5a9e08e1fd0efa5844440704

[KyleAMathews]

It seems like we could avoid having to do a v2 release — since the new release would just add favicon support right? It wouldn't change anything about the existing manifest support?

I don't know anything about the webpack favicon plugin. I think gatsby-plugin-sharp would work great for generating different icons so that's definitely an option. But depending on how easy the webpack plugin is, it could be a lot less work using that.

One advantage of gatsby-plugin-sharp is it'll keep the same API for gatsby v2 whereas we'll be doing a major upgrade of webpack so don't know how the webpack plugin will update.

[KyleAMathews]

Thanks for tackling this! This will be huge — having a drop-in plugin that does everything right for you.

[moonmeister]

@KyleAMathews Yeah I think this could be a stellar plugin.

1. I'll assume by avoiding v2 release you mean that in a semantic way...as in, there's no breaking changes? The trouble is currently the icons part of the manifest will change from being manually configured to automatically populated. Old configs will cause issues by including things that are not needed and not include new info needed for the auto generation of that part of the manifest and the images. This isn't impossible to work around...it just adds complexity to the code and config...it's a decision we have to make.

2. Also, this adds favicon support but also IE and Apple support (apple is working on implementing the web app manifest spec...it's just not released yet). As well as automatic image generation.

3. the webpack favicon plugin is a nonstarter so we can forget about that (though lots of people have created variations that might be useful). At the end of the day I think trying to roll our own with gatsby-plugin-sharp will provide the best long term compatibility and sustainability.

I think the config issue is maybe one of the biggest challenges on this, maybe we can chat more about it on Friday, others might have good input.

[KyleAMathews]

The trouble is currently the icons part of the manifest will change from being manually configured to automatically populated.

Ah! Ok, yeah, it's a breaking change then :-) Not a huge deal.

But yeah, we can talk more about this Friday ;-)

[moonmeister]

So, someone pointed out to me over at Creatiwity/gatsby-plugin-favicon#20 that if you disable the "chrome" on gatsby-plugin-manifest you can then also use this manifest plugin in it's current state. You still don't have have auto generated images but all the weird backwards compatible stuff is already generated then. So, in the name of simplicity and time I'm going to scope my own work to focus on simply adding a single boolean to this that auto generates the images and populates the icons JSON.

[gatsbybot]

Deploy preview for gatsbygram ready!

Built with commit 5707361deae2395461e9b9221ff28e9613d1bd7c

https://deploy-preview-4382--gatsbygram.netlify.com

[moonmeister]

@KyleAMathews I got images being generated, and I figured out what I think is a pretty slick way to handle stuff so none of this is a breaking change. The problem is half the images are generate then it segfaults. :(

The issue is sharp is running async and so it's not waiting for all the jobs to be returned before moving on in the build. So, how do I make it wait? I've never touched Promise before and I'm thinking that's helpful but I've tried several ways of using promise as shown in examples and it doesn't fix the issue.

The one thing that has made the segfaults go away is moving the image generation out of the function and into the primary code (see below) however no images ever get created...at least with the segfaults I got a couple images...

Please point me in the right direction, Thanks!

const fs = require(`fs`)
const Promise = require(`bluebird`)
const sharp = require(`sharp`)

// default icons for generating icons
const defaultIcons = [
  {
      "src": `icons/icon-48x48.png`,
      "sizes": `48x48`,
      "type": `image/png`,
  },
  {
      "src": `icons/icon-72x72.png`,
      "sizes": `72x72`,
      "type": `image/png`,
  },
  {
      "src": `icons/icon-96x96.png`,
      "sizes": `96x96`,
      "type": `image/png`,
  },
  {
      "src": `icons/icon-144x144.png`,
      "sizes": `144x144`,
      "type": `image/png`,
  },
  {
      "src": `icons/icon-192x192.png`,
      "sizes": `192x192`,
      "type": `image/png`,
  },
  {
      "src": `icons/icon-256x256.png`,
      "sizes": `256x256`,
      "type": `image/png`,
  },
  {
      "src": `icons/icon-384x384.png`,
      "sizes": `384x384`,
      "type": `image/png`,
  },
  {
      "src": `icons/icon-512x512.png`,
      "sizes": `512x512`,
      "type": `image/png`,
  },
]

//sharp.simd(true)

function generateIcons(icons, srcIcon) {
  Promise.map(icons, icon => {
    const size = parseInt(icon.sizes.substring(0, icon.sizes.lastIndexOf(`x`)))
    const imgPath = `./public/` + icon.src
    
    return sharp(srcIcon)
      .resize(size)
      .toFile(imgPath)

  })
}

exports.onPostBuild = (args, pluginOptions) =>
  new Promise(resolve => {
    console.log(`starting image stuff`)

    const { icon } = pluginOptions
    const manifest = { ...pluginOptions }
  
    delete manifest.plugins
    delete manifest.icon

    if (!manifest.icons) {
      manifest.icons = defaultIcons
    }

    const iconPath = `./public/` + manifest.icons[0].src.substring(0, manifest.icons[0].src.lastIndexOf(`/`))

    if (!fs.existsSync(iconPath)){
      fs.mkdirSync(iconPath)
    }
    
    generateIcons(manifest.icons, icon)

    fs.writeFileSync(`${iconPath}/manifest.json`, JSON.stringify(manifest))

    console.log(`ending image stuff`)

    resolve()
  })

[KyleAMathews]

You'd need to return Promise.map from generateIcons. Then to handle the promise generateIcons returns, you'd write generateIcons().then(() => { // put remainder of code here })

[gatsbybot]

Deploy preview for using-drupal ready!

Built with commit 07f8c96

https://deploy-preview-4382--using-drupal.netlify.com

[moonmeister]

@KyleAMathews Okay, I've fixed the branch I blew up...should all be good. I've implemented the auto image generation and it's all backwards compatible. Let me know if there are any improvements that need to be made. Other than that I need to probably test some various configurations and make sure all is happy. Thoughts?

Checkout the docs updates I made to see an explanation of the config changes and how it functions.

@shannonbux Any input on my doc changes?

Thanks

[moonmeister]

@KyleAMathews @shannonbux Hey, FYI I'm taking ~5 months off starting on April 28th to go hiking in the woods from Mexico to Canada on the Pacific Crest Trail. If there is anything you want done on this PR the sooner you can let me know the better chance I'll be able to get it done before leaving. Thanks.

[KyleAMathews]

Looking really good! Could you change gatsbyjs.org (www) to use this now instead of our manually created icons! That'd be a sweet improvement :-)

[KyleAMathews]

Does including a "hybrid" configuration make sense? Wouldn't you just do one or the other? Or is the idea that you could want different icons at larger sizes?

[moonmeister]

Yeah. It was going to be more code to eliminate this option then it was to leave it in. I figured it was useful if people wanted to not create all the icons or wanted to create different or larger icons. It's also useful if they just didn't like my naming conventions and wanted to modify path or name. I also believe it'd allow them to use webp or jpeg instead of png if they so desired (I believe the default output image is set to match the input. So if they gave it a jpeg it'd spit out a jpeg).

Basically it gives more options and without adding any code complications and little to no config complications.

[moonmeister]

@KyleAMathews Yeah, I can look into swapping over the gatsbyjs.org site. Do you want that on the same PR?

[shannonbux]

Very well-organized doc! Sounds intriguing and further motivates me to keep learning to code so I can use stuff like this!

[shannonbux]

I'd stay consistent with naming things, so either auto here and in the header below, or automatic in both cases. Not a huge deal, just makes it easier to scan the document for relevant information.

Might also make sense to stick with config or configuration and stay consistent.

[shannonbux]

Also, will people know generally what this plugin is about? A brief description before the how to use section could be handy.

[shannonbux]

What about also rewording the last sentence to read "These three configuration options are explained below. The plugin functions differently depending on which of the three you choose." This would clearly link this paragraph with the following paragraph, and linking is nearly always a great thing.

[moonmeister]

All sound good

[shannonbux]

It'd be really cool to see a 1-2 sentence use-case explained for each config option. Why would l choose manual vs. hybrid vs. automatic?

[moonmeister]

YES! And I'll reorder cause auto should be first since it'll be the default most people want.

[shannonbux]

"to be used to" caught my eye and I had to read it a few times (passive voice, which is necessary in technical writing sometimes). Is there a way to make it active? "this plugin will use the hi-res source icon you provide to auto generate your defined icons"? Hm, not sure that's clear. What I'm really curious about is what is a source icon? Will people know what that is?

Also a couple commas could be useful "...manifest, but instead of manually generating the defined icons, you..."

[moonmeister]

I rewrote this whole thing cause I reordered stuff

[shannonbux]

@moonmeister that hike is literally my dream. I want to hear all about it someday! Sounds incredible and hope you have a really nice time.

[moonmeister]

@shannonbux It's been a dream of mine for a while too! I thought of you the other day cause my flight down there goes through SLC. It's unfortunate you can't stow away and get on the trail too! I'll be posting on instagram, so follow me there. Also, this is why I can't make write the docs in pdx...cause I'll be hiking. :(

[moonmeister]

Well that was smooth, I managed not to hit save on the last set of changes...then pushed and closed out of the editor and clicked straight through the "Save" prompt...give me a sec...I have to rewrite a bunch of stuff.

[moonmeister]

alright, try that on for size. :)

[KyleAMathews]

Made some changes — lemme know what you think.

[moonmeister]

@KyleAMathews Overall I like the changes you made. I've cleaned up some more formatting and made some of the language more consistent to how you modified it using mode and such.

The big thing i rewrote the hybrid description. hybrid does not combine the default icons array and the one provided, it only creates icons from the array provided. I rewrote the section, hopefully it makes sense now. Let me know.

[KyleAMathews]

Ah! I didn't understand what the hybrid mode is doing clearly :-) Looking good now!

[KyleAMathews]

Thanks! This is a great improvement.

[KyleAMathews]

Oh, we didn't get this into gatsbyjs.org — you want to add that on a follow-on PR?

[moonmeister]

@KyleAMathews Yeah I'll try, it should be easy. I have a PR on Netlify CMS I should get done before I head out, so we'll see if I have time. I'll do what I can.

[ChrisBoon]

It looks like there was a typo in 'gatsby-ssr.js' for gatsby-plugin-manifest. The manifest link is missing a forward slash. It is currently:

   href={withPrefix(`manifest.json`)}

but should be

href={withPrefix(`/manifest.json`)}

Without this change it is creating the link as /PATH-PREFIXmanifest.json, not /PATH-PREFIX/manifest.json.

I can try and do a pull request, but it might be a couple of days before I get the time to go through all the steps.

[KyleAMathews]

@ChrisBoon PR would be great!

[moonmeister]

@ChrisBoon this is the Gatsby site and looks fine.

Am I missing something? What's your config look like?

[ChrisBoon]

The problem doesn't appear in the Gatsby site as that isn't hosted at a subdirectory and so doesn't use a path-prefix.

The issue is pretty simple. running withPrefix('') in develop mode or build mode without the --prefix-paths flag returns /, so withPrefix('manifest.json') will return /manifest.json as expected.

However, running withPrefix('') with the --prefix-paths flag returns /pathPrefix, so withPrefix('manifest.json') will return /pathPrefixmanifest.json.

Switching the function to withPrefix('/manifest.json')obviously resolves properly in --prefix-paths mode as it inserts the missing /. You might think this would cause a problem without '--withPrefix' as it looks like it would return //manifest.json. However the withPrefix function is elegant enough that it checks whether a forward slash is needed or not and it actually returns /manifest.json. eg withPrefix('') and withPrefix('/') both return / in develop mode.

Here are some console logs running in develop mode and build with the --prefix-paths flag:

These are produced by this code:

  console.log('withPrefix()');
  console.log(withPrefix(''));
  console.log("withPrefix('manifest.json')");
  console.log(withPrefix('manifest.json'));
  console.log("withPrefix('/manifest.json')");
  console.log(withPrefix('/manifest.json'));

[moonmeister]

@ChrisBoon Makes Sense. Thanks for the detail. If I have some time I can submit a PR or you're welcome to. Thanks.

[halkeye]

btw, totally fixed the prefix thing a couple weeks ago not noticing this ticket - #5406

[ChrisBoon]

@halkeye - Awesome, thanks for letting me know (and for fixing it 😀).

[jlengstorf]

Hiya @moonmeister! 👋

This is definitely late, but on behalf of the entire Gatsby community, I wanted to say thank you for being here.

Gatsby is built by awesome people like you. Let us say “thanks” in two ways:

1. We’d like to send you some Gatsby swag. As a token of our appreciation, you can go to the Gatsby Swag Store and log in with your GitHub account to get a coupon code good for one free piece of swag. (We’ve got t-shirts and hats, plus some socks that are really razzing our berries right now.)
2. If you’re not already part of it, we just invited you to join the Gatsby organization on GitHub. This will add you to our team of maintainers. You’ll receive an email shortly asking you to confirm. By joining the team, you’ll be able to label issues, review pull requests, and merge approved pull requests.

If you have questions, please don’t hesitate to reach out to us: tweet at @gatsbyjs and we’ll come a-runnin’.

Thanks again! 💪💜