Proposal: First Class Shortcode Support

[nickreese]

Hey Elder.js users.

I want to float my current thinking on shortcodes and the plan to add them to the Elder.js core to get buy-in from early users.

Benefit: Shortcodes make static content dynamic and future proof. Elder.js should support them natively.

The Context

Whether your content lives in .md files, on Prismic, Contentful, WordPress, Strapi, your own CMS, or elsehwere, content is generally pretty static.

That said, anytime content lives in a CMS there is always a demand to add 'functionality' to this content.

In my experience thisfunctionalities come in a few flavors.

• Adding custom HTML to style/wrap content or achieve design goals. (most common)
• Other times on data driven sites you want to make just a part of the static content dynamic replacing a datapoint with a real statistic. (see below)
• Embedding an arbitrary Svelte component.

Shortcodes to solve all of these problems.

Shortcodes

If you aren't familiar with shortcodes are strings that can wrap content or have their own attributes:

• Self Closing: {{shortcode attribute="" /}}
• Wrapping: {{shortcode}}wraps{{/shortcode}}

NOTE: The {{ and }} brackets vary from system to system.

1. Custom HTML Output

I'm in the process of porting my own website to Elder.js. During this process there are many times where I want a <div class="box">content here</div> to add design flair.

To achieve these I've built a simple shortcode: {{box}}content here{{/box}} allowing me to change the markup as needed should my needs change in the future.

2. Data With Shortcodes

A common use case for shortcodes that we've seen on ElderGuide.com and other properties that we are developing with Elder.js is the need to have an "article" with otherwise static content but that needs a datapoint from a database. (outlined above)

For example we often need to have content that would have this functionality: The US has [numberOfNursingHomes] nursing homes nationwide.

As it currently stands we do this type of replacement within our data functions and have a shortcode like so: {{stat key="numberOfNursingHomes" /}}.

The Problem:

As I've extracting out functionality from our sites, I'm finding more and more cases where plugins could offer shortcodes as well.

I've also found a major need for a shortcode to embed Svelte components, but the ideal implementation doesn't exist within the current Elder.js framework.

A. Minimizing Plugin Interdependence

My initial plan was to release a shortcode plugin using hooks. The shortcode plugin would register and manage the shortcodes from other plugins.

Then if a sister plugin (say the "image plugin") wanted to offer a shortcode it would require the shortcode plugin to also be installed.

The problem with this approach is plugin interdependence.

It just doesn't feel right for a plugin to only offer 1/3 of it's functionality without another plugin installed. There is also an issue of making sure each plugin doesn't overwrite the other's functionality.

B. data.key replacement

Another hurdle is that in a perfect world we'd be able to do {{svelteComponent component="Clock" props="data.key" options="{loading: "eager"}"/}} and the clock component would be hydrated with the values found on data.key.

In order to achieve this, the shortcode plugin would need the context of the data object returned from the route and would need to run between the generation of the route html and the template html.

Currently there is no hook that can support this... and hooks aren't really the right solution because we only want 1 instance of the shortcode parser to run.

The Plan

Our current implementations use a fork of the https://github.com/metaplatform/meta-shortcodes library. I've used this shortcode library in one form or another since 2016. It is battle tested across my sites, allows for open and close bracket customization, but could use a modernization effort. (not the focus of this proposal)

To implement shortcodes the plan is as follows:

Page.ts:

1. In Page.ts after the page.route.templateComponent function has built the route's HTML, shortcodes would be executed on the returned HTML.
2. After the shortcodes have executed, the returned html would then be passed into page.route.layout and page HTML generation would continue as it currently does.

The benefits of this location are 3 fold:

1. Whether content comes from a CMS, Svelte component, or a data store, we can assume that the shortcodes would be in the HTML returned by the route.
2. Because we still have to process the page.route.layout we can use Elder.js' internal system for inlineSvelteComponent to embed arbitrary svelte components offering a solution to embed Svelte components {{svelteComponent component="Clock" /}}.
3. Because all data related hooks and functions will have executed, we know that the data object is stable and could offer the ability to pass the data context into shortcodes as well allowing for {{svelteComponent component="Clock" props="data.key"/}} where data.key would be the key taken from the data object.

Defining Shortcodes:

Shortcodes could be defined in two places:

1. elder.config.js for user defined shortcodes.
2. plugins could define shortcodes by returning them during plugin initialization.

Proposed Definition (simple)

• props are the attributes defined on the shortcode.
• content is the content wrapped in the shortcode.
• data is the data object.

// elder.config.js

shortcodes: [
        {
          shortcode: 'box',
          run: (props, content, data) => {
            return `<div class="box">${content}</div>`;
          },
        },
]

Proposed Definition (robust)

The biggest limitation to the 'simple' design is there is no access to add css, js, or elements to the <head>.

I've hit this limitation in the past with WordPress and it was limiting, so a more complex design that would be more robust would be to support an API signature as below where css, js, and head are all added to the stacks needed to support them.

NOTE: this requires moving where stacks are processed.

// elder.config.js

shortcodes: [
        {
          shortcode: 'box',
          run: (props, content, data) => {
            return { html: `<div class="box">${content}</div>`, css:'', js: '', head:'' }
          },
        },
]

Shortcodes by Default

By default the only shortcode I'd imagine shipping is the {{svelteComponent component="Clock" props="" options="" /}}.

This does add a small regex call that appears to add about 1.2ms per page generation time in local testing.

---

This is the first major feature that I'm looking at adding to the core and I'd like community feedback. I'm not sure how OSS projects usually handle this but my goal is to get buy-in from early users. If you have feedback I'd love to hear it.

Having used several static site generators, I do believe this functionality belongs in the core. One of my biggest gripes with Gatsby is that it pushes all of the customization to plugins and solves the plugin interdependence problem with "Themes." While this works for one-off projects, it doesn't allow easy copy and pasting between projects because each Gatsby project is it's own special snowflake due to plugin interdependence. I think Elder.js offering shortcodes from the core is a wise move but would love feedback and some devil's advocates on why it shouldn't be in the core.

[nickreese]

@jbmoelker and @halafi would love your thoughts here.

[jbmoelker]

👏 nice writeup! I'm really not sure what the best approach would be, I'm not a big shortcode user myself, but I have some thoughts:

• Yes, I do think shortcodes may lower the barrier to entry.
• I would always go for an API signature that's easy to extend, so returning an object like in the robust proposal sounds like a smart thing to do.
• I would also change the parameter signature from run: (props, content, data) => { to run: ({ props, content, data }) => {. Makes it easier to use when you only need content or data.
• I'm not sure what would be the best way to register a shortcode. An array notation like you have now could silently overwrite one shortcode with another. Maybe an object notation makes more sense { myShortcode: { run: .... Or do what Elventy does: config.addShortcode('myShortCode', ....
• I'm not sure what would be best for the syntax for shortcodes. Since Svelte already uses a curly brace notation { ... }, maybe it's best to use a square bracket notation [ ... /]?
• Could/should there be more pre/post hooks for more fine-grained control, and would take allow for shortcodes?
• If you'd make the shortcode plugin part of the Elder core would that reduce the issue raised under "A. Minimizing Plugin Interdependence"?
• Another idea: should a shortcode template just be a Svelte component? So <script>export let props ...?

I think I need to sleep on this ;)

[nickreese]

@jbmoelker Awesome. Thank you for your thoughts, they helped spawn some ideas I'd never considered.

• Shortcode brackets can be customized. I found that [ caused issues with markdown so I ported all my WordPress shortcodes to {{ when reworking my personal site. Can you think of a usecase for Shortcodes in Svelte?
• Re: Plugin Interdependence: yep, by adding it to the core we can allow all plugins to use the same interface AND allow access to the route's data object / stacks which we can't currently do with the hooks.
• I like the cleverness of each shortcode being a Svelte component, but I think that fact that they are SSR only makes that pattern really confusing. helpers.inlineSvelteComponent(component, props, options) would need to be in scope for the shortcodes.
• As far as offering a hook for fine grained control. I'd rather push the tools for that control into the shortcode scope. For instance making sure request is in scope allows checking for request.route to have a shortcode return an empty string so nothing happens.

Adding Shortcodes:

I like the idea of a helper function, but I'm not sure what benefit it gives us over just accepting an object/array. I can see it being useful if Elder.js didn't have such a well defined bootstrap process. I'll dig into 11ty's prior art more.

Naming Conflicts:

• When building the shortcode parser, we can easily check for naming conflicts and throw an error/warning if there are.

Updated Signature:

I can lots of uses where we need helpers, query, plugin (when added by a plugin), and request within the shortcodes.

Further giving shortcodes access to many of the same props available on hooks we should push any fine grained control needs down to the shortcode level.

shortcodes: [
  {
      shortcode: 'box',
      run: ({ props, content, data, helpers, plugin, request, query }) => {
        return { html: `<div class="box">${content}</div>`, css: '', js: '', head: '' };
      },
    }
]

Notes:

Potential Speedup Available

The existing shortcode solution requires the shortcode parser to be rebuilt on each request to get the data and request objects into the context. I'm not sure what the speed impact would be, but obviously a rewrite of that dependency may mitigate this issue. (Going to look at how 11ty does it.)

Executing Shortcodes

I'm torn between offering a 'user only' hook and introducing that documentation complexity vs just offering a flag to enable/disable or provide an array of routes to run them on.

Async Support?

One problem is that I don't believe the current shortcode tool supports Async. Given shortcodes have access to the query object, this would make it trivial to make replaceable data point shortcode if you could make async database calls there.

Empty / Undefined Shortcode Return

We need plan for shortcodes to return undefined and see how that impacts the current implementation. I could see that breaking it.

Read Only Proxies

Not to self: We'll want all props of the shortcodes to be read only proxies.

[nickreese]

Shortcode support started: #35