Integrate documentation into frontend

[LukSteib]

Goal

Research and decide on best doc standardisation library

Context

Initial requirement:

Setup something like tsdoc (can be enforced via a linter rule)

Tasks

• Collect possible alternatives
• Propose qualities to make judgment
• compare, discuss and conclude

[valiafetisov]

Requirements

In general, there are at least two different problems

1. Provide self-explainable interface for querying data, for example:
   • 1A Graphql queries that have description visible in the graphql explorer
   • 1B Graphql explorer that have something like a swagger: an interface to execute known queries with the ability to add descriptions to them within the code
2. Provide self-explainable codebase for other developers who would want to extend it
   • 2A Enforce docstrings for all functions
   • 2B Build static documentation of the code

Ideally, I imagine that those those two things can be combined (ie via an extension), but if that's not possible, only the 2 should be done in this PR. In any case, the final outcome should be:

• [eslint] rules to enforce docstrings in our codebase
  • Comprehensive docscrings for all existing functions
• A command to build [single-page] html-based documentation of the code
• An integration into our existing playground/server setup
  • Served as a separate static page by the existing express server or extended/interlinked with the playground

Research tasks

• Research solutions for 1 and 2 separately
• Identify if 1 and 2 can be merged together in some way
• Research tsdoc alternatives, ie jsdoc seem to support typescript and is way more established
• Research available build templates, eg https://ankdev.me/clean-jsdoc-theme/v4/
  • Are there single-page templates or can any existing template be configured to be a single page or how the pagination works? Does it follow code structure?
• Research on how to integrate proposed solutions with graphql-playground-middleware-express or express itself
• Propose a concrete way forward

[valiafetisov]

During today's standup we've though if maybe docus can fulfil some of the outlined requirements?

[KirillDogadin-std]

WIP post

1

• There is a bunch of tools e.g. listed on https://nordicapis.com/how-to-autogenerate-graphql-api-documentation/
• since we're building our Gql on nexus, i would assume that there should be a documentation embedded, but sadly as of this issue it's not implemented yet.

2

i would prefer to use tsdoc over jsdoc since there's a rule of jsdoc that would become overly irritating - having to specify the types of variables and returns twice (jsdoc one being not needed since it's already covered by ts itself). The link about ts supporting js doc in this post refers to typing the js files and not documenting the ts code itself.

[valiafetisov]

Ok, so let's drop 1. from this issue, at least until powerhouse-inc/powerhouse#357 is resolved.

Requirement 2 becomes more practical: How to integrate documentation of the document-model-libs into our frontend. There are few options:

• Use markdown files they will generate and distribute with the npm package and render them via vue/nuxt
  • Example files: docs.markdown.zip
• Use html files they will generate and integrate it into our frontend
  • Example files: docs.html.zip
• Generate our own markdown/html/vue files from their source code and integrate it into our frontend
  • Or collect practical advices/package suggestions on how to make their generated docs more suitable for integration

Please note:

• It's important to serve documentation properly rendered on the backend (so it can be indexed by search engines)
  • If it's done via vue, ensure it's SSR rendered
  • Avoid iframes
• Style-wise, we would prefer to make it clean, github-like
• Structure-wise, I expect that we render single-page or same folder structure as provided under /docs/

[KirillDogadin-std]

1. markdown
   With nuxt 3 we're able to utilize the content directory (plugin) that is able to provide access to markdown files:

   • example
   • official docs 1, official docs 2, official docs 3
   • from what i can see in the links above:
     • + we have a raw data that is not styled, hence we should be able (like in the example link) apply or own styles to it
     • - we have to integrate styles. Otherwise the page looks like one from 2000th year.
       • should be compatable with https://github.com/sindresorhus/github-markdown-css because we can abstract the access to md files via nuxt's <ContentDoc /> component (this component is in the div with corresponding styles applied)
     • - the links in markdown are ... markdown-like. Which means: it assumes that the file is located in the FS and e.g. is opened in markdown viewer of vs-code app. So the links in this file are pointing to the files of FS and not the links in format like ./dir/SomeFile.md. I don't think it is 1-to-1 compatible with browser paths (i.e. in order to access file content/file.md via nuxt3 app we do use path /file in the browser). therefore: we need to write logic to process these transitions properly or change md files before adding them to content dir.

2. Html

   • that's a ready-to-go doc that could be added as static item to the nuxt. via public/ directory we could upload the html directory.
   • + if we like styles that are already there - that's all we would have to do.
   • - if we don't like the styles - that is more work to fix/adjust them.
   • + relative links are more compatible than md: .html extension is present in the filename, link itself and is required by nuxt public/

3. Documentation generation from the source code:

   • currently in the linked repo i did not see that there are doc comments in the src code.
     • it follows that we apparently (?) have to coordinate in order to agree on having these documentation comments there
     • the project is in ts, so i would imagine that having tsdoc or typedoc(https://typedoc.org/tags/defaultValue/) there is suitable. I find that docs can be generated via https://typedoc.org
     • among the themes we can choose the most suitable one. e.g. https://www.npmjs.com/package/typedoc-github-wiki-theme
   • + Always up to date and neglects the need for teams to collaborate when document model is updated (and docs are changed): it is possible to build a pipeline that grabs code from the original repo (main branch), generates code from it. Depending on the generation result (e.g. md or html) we apply one of the previous points.
   • - more time required since this option is the extension of the previous ones.
   • - / + inherits pros and cons of previous html and md approaches (depending on the configuration we choose here)
   • + it is possible to force the docs to be compatible with the code via some linter (eslint plugin? https://tsdoc.org/pages/packages/eslint-plugin-tsdoc/ ). So if this is set up, we also are more certain that the docs are compatible with the code.

4. Vue

• i did not discover a vue plugin for typedoc
• it seems that nuxt3 itself already handles md files good enough without any additional plugins (given that we also can style the md files in vue...)

---

with the proposed approach we can stick to the same doc structure or make the generated documentation into a single md file via https://www.npmjs.com/package/concat-md tool (script that gets the files in docs dir and passes them to this tool)

[valiafetisov]

it seems that nuxt3 itself already handles md files good enough without any additional plugins

Great, but:

• Does this allow us to style it using github-markdown-css?
• Does relative links work?
• Does this allow us to show md files on the main page? (I guess this is optional, as the main page can be custom and just link to the docs)

As discussed in the daily

• Please proceed to integrate provided docs into our frontend
  • Ensure relative links are working
  • Ensure docs are properly rendered with the suggested style
  • Ensure docs have the same header and layout as the rest of the website with correctly highlighted menu item "Documentation"
  • Please add link/main page itself to be shown inside rectangle at /
• Please provide clear integration requirements (e.g. a folder within npm package)
  • Any recommendations (e.g. a linter, a different markdown generator, etc)

[KirillDogadin-std]

Please add link/main page itself to be shown inside rectangle at /

this is not clear. please rephrase.

Please provide clear integration requirements (e.g. a folder within npm package)
Any recommendations (e.g. a linter, a different markdown generator, etc)

so in #42 there's the draft that approximates how this would look like if the docs are provided in the following manner:

1. generate them as it is done now.
2. process them via https://www.npmjs.com/package/concat-md like in the file https://github.com/makerdao-ses/switchboard-boilerplate/blob/ffdbb6da8de6a0e98e921f69d4c95788ffa153eb/frontend/scripts/generateDocs.ts#L1 in order to receive a single-page md document without file refs
3. put the generated file into e.g. docs/ directory of e.g. npm package.

• in case the draft PR does not display the desired approximation of the docs, lmk.

[valiafetisov]

Please add link/main page itself to be shown inside rectangle at /

this is not clear. please rephrase.

I think you already provided a link to the docs, that by changing the link under documentation button in the header

draft that approximates how this would look like if the docs are provided in the following manner

• So how do you suggest to provide the docs to us? As part of the npm package/as a zip file/etc?
• Does current process work inside the docker image automatically? And will it work when provided as the npm package?
• Does current process remove repetitive Table of contents manually on our side?
  • Do you suggest to remove those on their side using some parameters of the markdown generator?
  • Do you suggest to change/improve something on their side?
• Can you describe on the high-level what current realisation expects from the provided docs folder? Eg do we support images/code blocks, downloadable files (eg json example), other features, do we expect to only have markdown files in a specific format?

[KirillDogadin-std]

So how do you suggest to provide the docs to us? As part of the npm package/as a zip file/etc?

as in (3): single markdown file that has been generated via concat md. in e.g. docs/ directory of the npm package (as part of it)

Does current process work inside the docker image automatically? And will it work when provided as the npm package?

the current process immitates the file being provided and moved to the content/ dir. Which means:

• in the current as it is now works within docker
• the draft yet has to be adjusted to grab the file from the specified location instead of default 'content dir'. After we agree on the way we get the file - i will adjust accordingly.
• if it is provided as a part of the npm package, then there're multiple potential ways of settings this up:
  • copy the file over into the contents dir
  • provide load the contents up from the package and provide it to the displaying component

Does current process remove repetitive Table of contents manually on our side?

• did not find a nice way of doing this.
• it is simplier if it is not generated at the first place
  • https://www.npmjs.com/package/typedoc-plugin-markdown allows to use `--hideInPageTOC
• my personal preference is still to repeat the TOC:
  • it is as of now seems like an overhead to support mobile design fully
  • hence sider toc clutters the mobile layout and should maybe disappear?
  • presense of the in-text toc still allows to navigate on mobile.

Can you describe on the high-level what current realisation expects from the provided docs folder? Eg do we support images/code blocks, downloadable files (eg json example), other features, do we expect to only have markdown files in a specific format?

For now as i see nothing except for a single md file generated by the concatination is needed in this provided folder.

• since the documentation is generated by typedoc from tsdoc, i would not expect any images or downloadable files.
• if codeblocks means "markdown's codeblock rendering" then only the simplistic version , without language-specific highlights.

[KirillDogadin-std]

Summary of how the documentation has to look like.

1. Format: single markdown file, no table of contents
2. Contents: Md-Text, external URLs, md-links, files/images via absolute urls. I.e. ![](http://....)
   • can figure out and support for relative urls to files on demand
3. Uploaded to: npm package along with the other resources (source code)
4. Location: /markdown/ as right now https://github.com/acaldas/document-model-libs/tree/docs it is here.

---

Suggested adjustments to the current way of sharing this (in https://github.com/acaldas/document-model-libs/tree/docs)

1. Do not generate table of contents.

So the generation process looks approximately like

$ typedoc src/index.doc.ts --plugin typedoc-plugin-markdown --hideInPageTOC --out markdown

[valiafetisov]

since the documentation is generated by typedoc from tsdoc, i would not expect any images or downloadable files.

Tsdoc/jsdoc usually takes existing markdown files that are already in your codebase into the docs folder, so you can write introduction/general instructions to some subfolders/modules. It also supports full markdown syntax in the comments including images, I believe

my personal preference is still to repeat the TOC

I wouldn't agree with that. We should aim to have a nice mobile menu for that case (but not necessarily in the same PR as the current one)

Format: single markdown file, no table of contents
Concatenate documentation into a single file

Would rephrase my question: why do you suggest for them to use our script instead of us? It adds communication overhead and also complicates the process if we need to edit/improve/fix that script. I don't see a benefit of asking them to run it instead of us. But I see the benefit of them adding --hideInPageTOC to their pipeline

no images

Is it something that we can't support in principle or just a current limitation?

[KirillDogadin-std]

Tsdoc/jsdoc usually takes existing markdown files that are already in your codebase into the docs folder, so you can write introduction/general instructions to some subfolders/modules. It also supports full markdown syntax in the comments including images, I believe

👍🏻

I wouldn't agree with that. .. (but not necessarily in the same PR as the current one)

fine for me.

Would rephrase my question: why do you suggest for them to use our script instead of us? It adds communication overhead and also complicates the process if we need to edit/improve/fix that script. I don't see a benefit of asking them to run it instead of us.

Ok, if we ask to --hideInPageTOC, then the rest can be handled on our side.

Is it something that we can't support in principle or just a current limitation?

for the format like in github ![]() it displays the following:

Images are not formatted or centered. Technically speaking we could display them, but this is kind of unpredictable (but won't grow bigger than the html/vue container ofc)

If the example is actually fine, then i will edit the #22 (comment)

Since images are essentially attached files: same goes for them. As long as the link to the file download is funcitonal, it's fine. But there's obviously no guarantee that this is the case on our side since we don't store the files ourselves / validate their presence.

[valiafetisov]

As long as the link to the file download is funcitonal

To make funcitonal a little bit more technical, I would specify that we currently only support images that use absolute urls and not relative paths, but can figure it out if that's required

[KirillDogadin-std]

adjusted #22 (comment)

[valiafetisov]

To summarise,

How we integrated the documentation

• We've used provided zipped folder with markdown files
• We've build a single markdown file to have nice single-page documentation, searchable with CMD+F
• We've automatically build table of contents on our side
• We support all markdown features (but currently images have to have full urls)

Integration requirements

1. When generating markdown files, let's not include Table of contents, since we already handling it
   • Technically this can be done by providing additional flag --hideInPageTOC to the build command you're using
2. Please package markdown output into the npm package, so we're 100% sure that we're displaying documentation of the same version of the package we're using internally

[acaldas]

To summarise,

How we integrated the documentation

• We've used provided zipped folder with markdown files
• We've build a single markdown file to have nice single-page documentation, searchable with CMD+F
• We've automatically build table of contents on our side
• We support all markdown features (but currently images have to have full urls)

Integration requirements

1. When generating markdown files, let's not include Table of contents, since we already handling it

   • Technically this can be done by providing additional flag --hideInPageTOC to the build command you're using

2. Please package markdown output into the npm package, so we're 100% sure that we're displaying documentation of the same version of the package we're using internally

Hey,
The markdown has been added to the output under dist/docs/. Also added that flag to remove the TOC.