ROADMAP.norg

@document.meta
title: Neorg Query
description: A database for your notes with an eventual query language
authors: benlubas
categories: [
    projects
    readme
]
created: 2024-11-27T15:53:11-0500
updated: 2025-01-20T20:33:08-0500
version: 1.1.1
@end

* neorg-query

  (Planned) Features:
  - (x) Persistent LibSQL database that tracks file metadata for each file
  - (x) Parse files with rust-norg parser
  - (x) Update the database on launch
  -- (x) store file modified dates probably
  -- (x) remove files that are deleted
  -- (x) add/parse files that are new
  - (x) Watch current workspace and update on file changes
  - (x) `#sql` tag that populates on `:Neorg query run`
  - (x) format strings
  -- (x) modifiers
  -- ( ) auto format based on available fields
  - (x) Task support
  -- (x) `tasks` table
  -- (x) this could noticeably increase the parse time
  --- it really didn't add much
  -- ( ) Tweak the way that tasks dates are parsed
  --- ( ) support `Jan 1 2025` style of dates, there are easy to detect, and they're
          so much more natural for an American to write
  --- ( ) currently we fill in missing parts of the date with that potion from the
          current date. Behavior should be different for different fields
  ---- ( ) Year: year of the next occurrence of that date, if on Jan 20 2025 we see
           a new tasks labeled `1st Jan` the year should probably be 2026
  ---- ( ) Month: similar, if we see `5th` and it's the first, use the current month,
           it's it's the 6th, use the next month
  ---- ( ) Hour, Min, Sec, set to 0
  -- ( ) Recurring tasks; we don't provide any support for these right now.
  --- there's a lot of work to do to properly support recurring tasks. I'm putting
      that off for now
  - ( ) Potentially provide SQL views, and or provide a way for people to define
        their own in configuration
  - ( ) improve multi threaded parsing performance
  - ( ) Keep track of links
  - ( ) Possible optional plaintext support for vector search and specific paragraph
        retrieval. This would increase the size of the DB by a lot

** Custom Query Building
   Eventually, we'd like to create a custom query language that makes it easier to
   construct queries. I've begun to look pat how obsidian queries and data view
   queries work, but still definitely want to talk more about this.

   Instead of a `#sql` tag, these will use `#query`, or even `#task` if we create
   a simplified query language for tasks.

*** Pros and Cons of SQL
    *Pros:*
    - Implementation is super simple. We literally just take the raw query from the
      user, and run it. No syntax parsing, no nothing.
    - Familiarity, most all devs have used SQL before

    *Cons:*
    - overly verbose
    - sql queries in general have some weird design flaws
    - joins are annoying to write
    - `path LIKE '%/folder/%'` is at least 6 too many chars to communicate that I want
      things in the `folder` folder
    - recursive data structures such as task trees don't work well with SQL, we need
      to do lua processing, which doesn't allow for easy user facing format
      functionality