[inbox] What is a MarkdownDB?

[rufuspollock]

It is two things

• The pattern itself which itself is two things (?!)
  • Using markdown files as a database
  • Extracting structured information from markdown files
• The library: for creating a database from markdown files ...
• Different phrases to illustrate
  • This is a MarkdownDB (i.e. a contentbase following the markdowndb pattern)
  • This is built using the MarkdownDB library (the library accessing a MarkdownDB)

Inbox of other material

• https://next.datahub.io/notes/markdowndb

Intro

A MarkdownDB is a pattern for treating markdown files as a lightweight database along with an API for accessing them.

More specifically it is a simple way to turn a collection of markdown files and their structured data (frontmatter, tags etc) into a queryable SQL database and API. Extracted metadata includes simple things like frontmatter, tags like #mytag and much more.

MarkdownDB is especially appropriate for ...

• Collections where the individual records include both rich text and structured metadata
• "Micro" collections with less than 10k records

The Pattern: markdown files are records

Database consists of two things:

• The data itself - in markdowndb these are markdown files
• An index and API for accessing and querying those files - in markdowndb this is an sqlite database

Roughly:

• Each markdown file corresponds to a record
• Each directory corresponds to a table (if you want it to)

Let's have an example:

my-random-file.md
movies
  return-of-the-jedi.md
  ...

return-of-the-jedi.md

---
date: 1983
budget: 32.7
---

# Return of the Jedi

Return of the Jedi (also known as Star Wars: Episode VI – Return of the Jedi) is a 1983 American epic space opera film directed by Richard Marquand. The screenplay is by Lawrence Kasdan and George Lucas from a story by Lucas, who was also the executive producer. The sequel to Star Wars (1977) and The Empire Strikes Back (1980), it is the third installment in the original Star Wars trilogy.

Architecture

Related efforts

What's distinctive about markdowndb approach (🚩 this is where pattern and library intermingle a bit - in the most general sense markdowndb would just be a way of treating markdown files as data)

• Do one thing only (and well): focused on building the index and providing an API
  • Does not get involved in render pipeline (as contentlayer.dev, etc
• extract structured content beyond frontmatter
• sql(ite) oriented (why reinvent the wheel!)
• not tied to a particular stack (in contrast to nuxt content, astro etc)
• open source (in constrast to tina content layer)

Inspirations etc

• contentlayer.dev
• https://content.nuxtjs.org
  • really nice
  • use sqlite (rather than mongo syntax)
  • tied to nuxt
  • gets involved in the rendering pipeline in a variety of ways