[Proposal] Provider DSL

[EisenbergEffect]

Problem

Having gone through the process of implementing a significant number of components for our library, writing documentation, and working with a 3rd party to create a tutorial for FAST, a number of "rough edges" with some of our approaches have popped up:

Using multiple design systems on the same page is challenging

Today, if you import from both fast-components and fast-components-msft you will experience a runtime error. This is due to the fact that both libraries register web components under the same element names. We did this to enable easily swapping the design system, under the goal of having a single set of components with multiple design systems. However, under the current implementation, the template and styles for a component are still statically linked to the custom element definition (and name) itself, preventing us from having both design systems present.

Element registration is a side-effect of importing

The underlying call to the customElements.define API in the platform is not predictable. It happens as a result of importing a component because the decorator on our components calls the define API during module evaluation. While this is a desirable convenience when building site or app-specific components, it's a problem for customers who want to consume a 3rd party library of components (e.g. fast-components).

There's no way to disambiguate element definitions

Due to the fact that element registration is a side-effect of importing, there's no opportunity to change the registered element names. This means there's no way to disambiguate our elements from another library that might happen to use the same names. Furthermore, there's no way to disambiguate between multiple design systems built on our same foundation, as described above in the issue on "Using multiple design systems on the same page is challenging".

We have an awkward need for seemingly inert expressions just to prevent tree shaking

Also due to the fact that element registration is a side-effect of importing, bundlers get confused about what the actual dependencies are and will tree-shake away all our components unless they are referenced directly in code. As a result, we're forced to do things like this:

import { FASTAnchor } from '@microsoft/fast-components';

FASTAnchor; // very strange!

Our design systems add an additional layer of inheritance just to associate styles

As mentioned above, we currently statically associate styles and templates with the element definition of an element. We're using the customElement decorator to do this, which requires a unique class. However, FASTElement has an imperative API that would allow us to make the association without needing the extra layer of inheritance. As a result of our current approach, we also end up with a number of classes in our API docs that have no API. They are just there to inherit from a base class so that styles can be associated.

There's no way to dynamically change an element's template or styles (or shadow dom mode)

Our API allows us to add or remove additional styles and behaviors, but not change out the base styles dynamically. Templates and styles are statically bound to the component and unchangeable. This prevents several interesting scenarios such as:

• Implementing layouts/master-pages for a router via changes to the router's template.
• Implementing different card designs, independent of content, without requiring different card elements
• Easier one-off component template or style alterations without being forced to define a whole new element
• Swapping templates based on runtime conditions (e.g. changes in available screen space, user preference changes, etc.)

Proposed Solution

To address the above issues, and open up additional possibilities, I'm proposing a number of changes across our packages. Below I'll address each package with some basic code samples and try to explain how all the pieces fit together to address the above issues and open up new possibilities.

Changes in fast-element

The main changes in fast-element are as follows:

• Change the timing of template instantiation and style adoption so that it happens in the connectedCallback rather than in the constructor
• Introduce a slight bit of indirection so that FASTElements can control template and style resolution on a per instance basis.

Underlying FASTElement is the Controller class, which controls template instantiation, binding, style adoption, lifecycle callbacks, attribute change notifications, dynamic behaviors and more. Presently, when the Controller is instantiated for an element, in its constructor, it immediately gets the template and styles from the element definition and applies them. Because the connectedCallback of an element happens before the first paint of a custom element, it's safe to delay the application of template and styles to this stage.

Reminder: The element definition is a normalized version of the configuration object that is passed to the customElement decorator.

In addition to delaying the application of template and styles, I propose adding some indirection so that that the Controller calls methods on the element, asking it to resolve its template and styles. By default, the base implementation would retrieve the template and styles from the element definition, but because these methods now exist, sub-classes can override this behavior to provide custom resolution logic. This is a critical extension point leveraged below with the introduction of FoundationElement.

Here's some psuedo-code that demonstrates the changes I propose to apply:

export class Controller {
  constructor() {
    // remove template and style application from here
  }

  onConnectedCallback() {
    // apply template and styles the first time the component is connected
    const template = this.element.resolveTemplate(); // indirection
    this.setTemplate(template); // a new method on Controller to set the template at any time

    const styles = this.element.resolveStyles(); // indirection
    this.setDefaultStyles(styles); // a new method on Controller to set the default styles at any time
  }
}

export class FASTElement {
  protected resolveTemplate() {
    // the default implementation preserves the present behavior
    // which gets the template from the definition
    return this.$fastController.definition.template;
  }

  protected resolveStyles() {
    // the default implementation preserves the present behavior
    // which gets the styles from the definition
    return this.$fastController.definition.styles;
  }
}

Note: A positive side-effect of moving the timing of template instantiation is that it would now be technically possible to combine the template instantiation and data binding operations into a single operation, possibly yielding additional performance improvements in startup time.

Changes in fast-foundation

The changes in fast-foundation build on the new extension points added in fast-element as well as introduce additional patterns for design systems. In particular, a DSL for design system creation is introduced. This system underlying the DSL also provides primitives upon which features such as advanced dependency injection could be built as an optional plugin. The main changes are as follows:

• Introduce FoundationElement, derived from FASTElement which will serve as a base class for elements with design system "smarts" as well as template and css dynamism.
• Introduce the general notion of a Provider
• Introduce a DSL for defining providers
• Introduce a helper for creating component registrations

The first step is to tap into the new capabilities of FASTElement. To do this, we'll introduce a new class called FoundationElement. Below is pseudo-code demonstrating what it adds on top of FASTElement:

export class FoundationElement extends FASTElement {
  @observable template;
  templateChanged(oldValue, newValue) {
    this.$fastController.setTemplate(newValue);
  }

  @observable css;
  cssChanged(oldValue, newValue) {
    this.$fastController.setDefaultStyles(newValue);
  }

  connectedCallback() {
    this.$fastProvider = this.resolveProvider();
    super.connectedCallback();
  }

  protected resolveTemplate() {
    return this.template
      || this.$fastProvider.resolveDefaultTemplateFor(this)
      || super.resolveTemplate();
  }

  protected resolveStyles() {
    return this.css
      || this.$fastProvider.resolveDefaultStylesFor(this)
      || super.resolveStyles();
  }
}

The first things that are added are two new @observable properties: template and css. These properties allow for any FoundationElement instance to have its template and base styles changed at any time. With the delay in template instantiation introduced in FASTElement this means that these values can be set through the template engine's binding system without incurring any cost due to accidental pre-rendering of stale templates.

The nex thing to note is that the connectedCallback is enhanced so that it resolves a provider. More on this is described below. For the purpose of this part of the explanation, the provider is an object that can provide templates and styles for custom elements. It is resolved through the DOM hierarchy.

Finally, the new methods resolveTemplate and resolveStyles which were introduced in FASTElement are overridden so that they access the template and css properties first, then the provider, in order to obtain templates and styles, before falling back to the base implementation.

The additions introduced in FoundationElement enable decoupling the element defined with the platform from its template and styles, which can now be dynamically resolved through observable properties or a parent DOM node (the provider).

Moving on to the Provider, it is a custom HTMLElement with an API something like this:

export class Provider extends FASTElement {
  readonly parentProvider: Provider;
  resolveDefaultTemplateFor(element): ElementViewTemplate | null;
  resolveDefaultStylesFor(element): ElementStyles | null;
}

The provider is a custom element that provides various data and services to descendent nodes. This can include design system properties, a dependency injection container, or any number of capabilities. For the purpose of this part of the explanation, we'll say that it minimally provides the ability to resolve templates and styles for an element. It also has a reference to its parent provider, as certain requests may be forwarded up the DOM tree to parent providers.

Note: Since the majority of apps will only use one design system (meaning Fluent or Material but not both), the provider system will keep a DOM-independent reference to the first defined provider. This will allow FoundationElement instances to take a fast path of directly accessing the provider instance without walking the DOM, for the purpose of resolving templates and styles only. This nuance is not shown in the pseudo-code above but will be an optimization in the actual implementation.

The next question to address resolves around the definition of the provider itself. For this purpose, I propose a DSL. The DSL starts with a Configuration object that is used to define the various aspects of a design system, in terms of our primitives. This includes templates, styles, design system properties, etc. It also provides a way to package up "registries" that encapsulate reusable configuration settings that can be applied to a provider. Finally, it provides a way to instantiate a Provider based on the definition contained in the Configuration object. The result of the defineProvider operation is that a new custom HTML element is defined, which can then be used in the DOM to provide all data and services needed. Here's some pseudo-code that shows what the Configuration and Registry APIs might look like:

interface Registry {
  register(config: Configuration);
}

export class Configuration {
  public readonly prefix: string;

  constructor(options = { prefix: 'fast' }) {
    this.prefix = options.prefix;
  }

  registerDesignSystemProperty(options): this;
  registerProviderAttribute(options): this;
  registerProviderStyles(styles): this;

  registerElement(Type, definition): this {}
  registerDefaultTemplate(elementName, template): this;
  registerDefaultStyles(elementName, styles): this;
  register(...registrations: Registry[]): this;

  defineProvider({ prefix }): typeof Provider;

  public static forComponent(options) {
    return function({ name?, template?, styles?, prefix? }) {
      return {
        register(config: Configuration) {
          const definition = {
            name: name || `${prefix || config.prefix}-${options.baseName}`
          };

          config
            .registerElement(options.type, definition);
            .registerDefaultTemplate(definition.name, template || options.template);
            .registerDefaultStyles(definition.name, styles || options.styles);
        }
      };
    }
  }
}

Take note of how the Configuration.forComponent(options) method takes a set of custom element metadata and produces a function that can create a Registry for that element. How these APIs are used in practice will be come clearer below when we look at how fast-components would change to use them.

Example changes in fast-components

With the above primitives, we can now define a design system more concretely in our code. To do so, we would inherit from the Configuration object. In the constructor, we would call APIs to define design system properties, attributes, styles and behaviors that are required statically by the design system.

export class FASTConfiguration extends Configuration {
  constructor(options) {
    super(options);

    // register design system properties
    // register provider attributes
    // set the provider styles/behaviors
  }
}

We would then create registry factories for each component that we want the design system to support. Here are some examples of what that would look like:

export const anchor = Configuration.forComponent({
  type: Anchor,
  baseName: 'anchor',
  template: AnchorTemplate,
  styles: FASTAnchorStyles
});

export const button = Configuration.forComponent({
  type: Button,
  baseName: 'button',
  template: ButtonTemplate,
  styles: FASTButtonStyles
});

These technique allows each component's registration with the design system to be modular, maintaining the ability to tree shake unused elements. Additional registries can be created for other shakeable items, such as behaviors, appearance capabilities, etc.

Now, let's take a look at how a customer uses this:

Example usage

import { FASTConfiguration, anchor, button } from '@microsoft/fast-components';

new FASTConfiguration()
  .register(
    anchor(),
    button()
  ).defineProvider();

The code above imports the design system configuration, along with the modular pieces that the customer wishes to use. Each piece is registered with the configuration and then the provider is defined.

When the provider is defined, it first creates a new custom element for the provider, defined with the platform, dynamically built from all the configuration settings. It then walks through the components and registers them one by one, along with their templates and styles.

In HTML, usage looks like this:

<fast-provider>
  <fast-anchor></fast-anchor>

  <fast-provider>
    <fast-anchor></fast-anchor>
  </fast-provider>
</fast-provider>

The template and styles for fast-element are derived from the singular fast-provider that is defined, while the design system properties are inherited through the DOM based on the closest instance of fast-provider.

We can also now define multiple providers for different design systems and use them together on the same page, without needing different element names (except for the provider):

import { FASTConfiguration, anchor, button } from '@microsoft/fast-components';
import { MSFTConfiguration, anchor as msftAnchor, button as msftButton } from '@microsoft/fast-components-msft';

new FASTConfiguration()
  .register(
    anchor(),
    button()
  ).defineProvider();

new MSFTConfiguration()
  .register(
    msftAnchor(),
    msftButton()
  ).defineProvider({ prefix: 'msft' });

Here's some HTML using this:

<fast-provider>
  <fast-anchor></fast-anchor>

  <fast-provider>
    <fast-anchor></fast-anchor>
  </fast-provider>
</fast-provider>

<msft-provider>
  <fast-anchor></fast-anchor>
</msft-provider>

Note that we can use the same fast-anchor element because when it has a different providers, its template and styles can then be derived based on which providers it is a descendent of.

Important: Our single file script concatenated builds would include code to create the provider instance and register all components by default so that using that script does not require writing any JavaScript.

Bigger Differences between Design Systems

Let's take the case of badge where the -msft version differs not only in styles, but also adds additional properties to the badge element. How do we handle this? There are several options, one way is to handle it the same way that we do today. Here's some code for that:

export class MSFTBadge extends Badge {
    @attr({ mode: "fromView" })
    public appearance: BadgeAppearance = "lightweight";
    private appearanceChanged(
        oldValue: BadgeAppearance,
        newValue: BadgeAppearance
    ): void {
        if (oldValue !== newValue) {
            DOM.queueUpdate(() => {
                this.classList.add(newValue);
                this.classList.remove(oldValue);
            });
        }
    }
}

export const badge = Configuration.forComponent({
  type: MSFTBadge,
  baseName: 'badge',
  template: BadgeTemplate,
  styles: MSFTBadgeStyles
});

How would the provider system work in this case?

• If there is a single defined provider, you will have a fast-badge with the above functionality.
• If two or more providers are defined, then the customer will need to disambiguate on the element name because we have a difference in behavior between the two elements. Without disambiguating, when the providers are defined, they will fail fast with an informative error message. To disambiguate, you would use provider code as follows:

import { FASTConfiguration, badge } from '@microsoft/fast-components';
import { MSFTConfiguration, badge as msftBadge } from '@microsoft/fast-components-msft';

new FASTConfiguration()
  .register(
    badge(),
  ).defineProvider();

new MSFTConfiguration()
  .register(
    msftBadge({ prefix: 'msft' }),
  ).defineProvider({ prefix: 'msft' });

While using inheritance to address this issue of variance it possible as shown above, there are other options which do not require inheritance as well. We should look at each component separately to see whether we truly need to introduce new behavior that cannot be added through any of the compositional affordances of FASTElement or FASTFoundation. For example, something like a Material Design ripple effect can actually be added through a Behavior associated with a style. Let's take a look at the case of badge above. Could that be implemented in an alternative way? Based on the current definition, yes. In fact, we don't need to add the appearance attribute at all because we don't have any real behavior that is being affected by that. It's only adding classes to the host for the purpose of styling. However, we can add host attribute selectors to handle this directly in CSS. For example, instead of this:

:host(.lightweight) {
    background: transparent;
    color: ${neutralForegroundRestBehavior.var};
    font-weight: 600;
}

We could do this:

:host([appearance=lightweight]) {
    background: transparent;
    color: ${neutralForegroundRestBehavior.var};
    font-weight: 600;
}

In fact, this is a "cheaper" implementation as it does not require all the platform machinery needed to define custom attributes, add their callbacks, and update host classes.

Regardless, we have a wide variety of techniques available to us to differentiate design-system specific variants of components. With the changes proposed here, we can actually achieve many more variations through runtime context, without requiring static solutions like inheritance and decorators (though these are always available to us if the situation requires them).

Other examples

Per Instance Style/Template Customization

const buttonTemplate = html`...`;
const buttonStyles = css`..`;

const template = html`
  <fast-button>Normal</fast-button>
  <fast-button template=${buttonTemplate} css=${buttonStyles}>Instance Override</fast-button>
`;

How Fluent UI Could Build on FAST

import { MSFTConfiguration, badge, MSFTButtonStyles } from '@microsoft/fast-components-msft';
import { Button, ButtonTemplate } from '@microsoft/fast-foundation';

export class FluentConfiguration extends MSFTConfiguration {
  constructor() {
    super({ prefix: 'fluent' }); // override default prefix

    // register additional design system properties
    // register additional provider attributes
    // change provider styles/behaviors
  }
}

// no changes
export badge;

// extend the button behavior
export FluentButton extends Button {
  @attr anAdditionalAttribute;
}

// compose additional styles
export const FluentButtonStyles = css`
  ${MSFTButtonStyles},
  ...some additional styles...
`;

// create the registration
export const button = FluentConfiguration.forComponent({
  type: FluentButton,
  baseName: 'button',
  template: ButtonTemplate,
  styles: FluentButtonStyles
});

import { FluentConfiguration, badge, button } from '@microsoft/fluent-components'; // made up package

new FluentConfiguration()
  .register(
    badge(),
    button()
  ).defineProvider();

<fluent-provider>
  <fluent-button>Click Me</fluent-button>
  <fluent-badge>Success!</fluent-badge>
</fluent-provider>

Could our React wrappers hook into this?

import { FASTConfiguration, anchor, button } from '@microsoft/fast-components';
import { wrapWithReact } from '@microsoft/fast-react-wrapper'; // not sure what we're calling this

const config = new FASTConfiguration()
  .register(
    anchor(),
    button()
  );

config.defineProvider();

const { FASTProvider, FASTButton, FASTAnchor } = wrapWithReact(config);

Advanced Dependency Injection

import { FASTConfiguration, anchor, button } from '@microsoft/fast-components';
import { Registration } from '@microsoft/fast-dependency-injection'; // made this up

new FASTConfiguration()
  .register(
    anchor(),
    button(),
    Registration.transient(HttpClient),
    Registration.singleton(ProfileService)
  ).defineProvider();

@inject(HttpClient)
export class ProfileService {
  constructor(private httpClient: HttpClient) { }
}

export class ProfileScreen extends FASTElement {
  @inject(ProfileService) profileService: ProfileService;
}

[awentzel]

I love it. I think this is very straight forward and intuitive just by scanning the code which will improve the onboarding experience.

[nicholasrice]

Thanks for putting this together @EisenbergEffect! Overall it looks great and super extensible.

A case I don't see addressed is inter-component dependencies. Lets say that component <fast-a> implements a template:

html`
<template>
  <fast-b>foobar</fast-b>
</template>
`

An author would need to know the implementation details of <fast-a>, specifically that it depends on <fast-b> being registered. With proper documentation this isn't necessarily a problem but it does seem difficult to use and reduces component portability.

Is this a case you've thought about? One potential solution that I see:

Expose the configuration from the provider?

If the configuration is exposed by the provider, each component could ensure any dependent component is registered. If it's not, then it could register it:

import { fastB } from "@microsoft/fast-components";

connectedCallback() {
  super.connectedCallback();

  this.$fastProvider.configuration.register(
    fastB() // Wouldn't want this to clobber any author-defined registration config
  )
}

Perhaps there is even a way to collect the tag-name from the provider for any dependent components used:

// This is mostly to illustrate the idea, not the implementation. I recall some issue about data-derived tag names...
import { fastB } from "@microsoft/fast-components";
const fastATemplate = html`
<template>
  <${x => x.$fastProvider.configuration.getTagNameFor(fastB)}>
    foobar
  </${x => x.$fastProvider.configuration.getTagNameFor(fastB)}>
</template>
`

Note: In general I think we would opt for composability and not a direct dependency on the component implementation. Perhaps that design pattern is enough to warrant the above un-necessary? Regardless, I think this case is common enough that we should have an answer for it.

[EisenbergEffect]

@nicholasrice I think we can handle the dependency issue without too much trouble. We could add that to the registration API so that component authors can declare that and we can auto-register those if not explicitly asked for (or maybe in the same way we can associate behaviors with css, we could associate elements with templates?).

The real trick is if the element names or prefixes are changed for a dependent component. That is tricky to handle without custom element registries, which we could use to ensure that the elements in the private shadow dom are always the ones we expect. I can imagine some tricksy ways to work around that. It's not as simple as the pseudo-code you have above, due to the way the template engine works, but we could come up with something. It will complicate this work though and might require some additional changes in fast-element to make happen (e.g. re-writing element names in templates during compilation or something like that). Do we need to take that on right now or could we push it out to later?

[EisenbergEffect]

An idea based on associating elements with templates:

const template = html`
  <foo-bar></foo-bar>
`.withRegistry({
  'foo-bar': FooBar
});

That wouldn't be needed for most templates, only those in component libraries that depend on other components. From this info, we can reverse map the element name used in the template to the actual name at runtime based on the provider configuration. I think this should also fit well with custom element registries, as this basically would define what should be in the registry associated with the shadow dom. For now, we can just handle this in our compiler.

[nicholasrice]

I think I like the .withRegistry() idea, but I don't quite follow the implementation. Are you saying that the template compiler would replace the "foo-bar" tag name with a "resolved" tag name (<foo-bar> in template => <bar-bat> in DOM) or are you saying that the compiler would register a foo-bar custom element if one did not already exist?

yea it's a bummer we don't have custom registries for this! I'd be okay with pushing the tag-resolution feature out if it makes it easier. This only concerns custom elements in shadow-dom so the tag-name doesn't really matter as long as it's the right element implementation, right?

[EisenbergEffect]

The first: we could teach the template compiler to remap the names I think. I'd prefer to delay doing this particular piece until we have someone come with a situation where we need it though, if we can. If we had more time to think about it, we could maybe come up with a way to do it as a custom registry polyfill of sorts.

[nicholasrice]

Another thought I want to capture here - there are cases where certain semantic elements are providers for CSS custom properties and should allow registration of property resolves. This includes components like cards, dialogs, flyouts, that have backgrounds that can (and often are) distinguished from the rest of the page.

[EisenbergEffect]

Dropping a note here that we want to see if we can lazily create a design system if one doesn't exist. The obvious problem is that in order to do that...we need to statically link everything into the lazy initialization code, which will prevent tree shaking. My thinking is that we have a function that is exported that does this. That could then be called in the single file script build or by a partner that wants to bundle things up.

Another note is that we want to enable customization of the element prefix even when a single file build is used. We could enable this with a data- on the script tag itself. For example:

<script type="module" src="..." data-prefix="fast"></script>

[EisenbergEffect]

A quick update on this work...

As part of the exploration, we decided to try directly incorporating a dependency injection system. The results of the exploration were very positive. So, we've reworked the entire provider plan on top of a core DI system. The linked PR contains the code for the DI, FoundationElement, and a configuration DSL for basic configuration of elements, templates, and styles. No components are yet swapped to the new system, as that would be a breaking change. We hope to merge the new system soon and then continue to evolve it so that it supports all aspects of the design system. Once that is in place, we'll migrate the components and publish a major version number bump.

[EisenbergEffect]

Closing this as "done" since the infrastructure for almost everything described is now in fast-foundation. Over the next month or two, we'll layer on additional design system capabilities and switch the components over to using the new system.