feat: add extensions support

[genericFJS]

Quick and dirty: Add "use" parameter which allows extensions (see #17 ).

Example usage (replaces token [[latex…]] with a colored span):
File App.svelte

<script>
  import SvelteMarkdown from "svelte-markdown";
  import LatexMarkdown from "./LatexMarkdown.svelte";

  const latexTokenizerExtension = {
    name: "latex",
    level: "inline",
    start(src) {
      return src.match(/\[\[/)?.index;
    },
    tokenizer(src, tokens) {
      const rule = /^\[\[latex\s*(?:color="(.*)")?\]\]/;
      const match = rule.exec(src);
      if (match) {
        console.log(match[1]);
        return {
          type: "latex",
          raw: match[0],
          text: match[0],
          color: match[1],
        };
      }
    },
  };

  const use = { extensions: [latexTokenizerExtension] };

  const renderers = {
    latex: LatexMarkdown,
  };

  const source = `
  # This is a header

This is a paragraph.

This is red [[latex color="red"]] text.

* This is a list
* With two items
  1. And a sublist
  2. That is ordered
    * With another
    * Sublist inside

| And this is | A table |
|-------------|---------|
| With two    | columns |`;
</script>

<SvelteMarkdown {source} {use} {renderers} />

Imported file LatexMarkdown.svelte

<script>
  export let color;
</script>

<span style={"color: " + color}>LaTeX</span>

[genericFJS]

Added a test and documentation.

[pablo-abc]

Since this can add renderers, if you're using TypeScript it would conflict a bit with the Renderers type above. Maybe extending said type with something like & Record<string, InstantiableSvelteComponentTyped> would work?

[genericFJS]

Since we don't use the renderer either way, maybe we should discourage passing one with the use parameter:

use?: Omit<marked.MarkedExtension, 'renderer'>[]

[genericFJS]

I just realized you were not talking about the use-type but the the Renderers type (just as you've written…). And there you are right, of course. The type would have to be added.

[pablo-abc]

Since we're using all of marked now, is there a difference on build size between importing Lexer vs doing new marked.Lexer()?

[genericFJS]

I did some rudimentary tests with the svelte template where I added a SvelteMarkdown component with an extension (as in the first comment of this PR):
There were no differences in files size (after yarn build) when using it as is VS removing the Lexer/Slugger imports entirely and writing

slugger = source ? new marked.Slugger : undefined

and

tokens = isInline ? marked.Lexer.lexInline(source) : marked.Lexer.lex(source)

Removing the imports seems to have no effect on build size (and could just be done for cosmetic reasons).

[pablo-abc]

This looks great! Thanks a lot! I just left a couple of questions

[genericFJS]

While testing a bit more I found a huge problem with my approach: by using marked.use I manipulate the global options. Marked therefore keeps the options (between tests). That means that I can not pass an extension in one test and have a following test where I expect the custom token not to be replaced.

[genericFJS]

I think we should close this pull request, since marked.use has a lot of side effects and implementing it side effect free seems too big a hassle.
Since marked.use transformes the extensions to another structure into the global marked.defaults options, it's easier if SvelteMarkdown does not support extensions directly, but the user may generate his own extension-enriched options and pass it as options to the SvelteMarkdown component:

<script>
  // [… as in first comment]

  // let marked save extension information to global options:
  marked.use({ extensions: [latexTokenizerExtension] });
  // use global marked options as parameter for SvelteMarkdown-component
  const options = marked.defaults;
</script>

<!-- works without this PR -->
<SvelteMarkdown {source} {options} {renderers} />

[genericFJS]

Closing in favor of manually doing it by passing the appropriate options.