Support for Tags

[lambtho12]

I just saw that you added "add support for #tags" in the backlog. This would be a fantastic feature indeed!
What do you have in mind for that? I also thought about it and was planning to make a feature request issue, but I could not find a way that would work with telescope. That being said, I am not a Telescope expert...

The features that would be awesome for me are mostly

• search files with some specific tags (grep style), but be able to combine multiple tags (not sure if this is possible)
• be able to explore existing tags and insert them on the note I am currently editing. Once again, not sure how that would be possible with telescope.

It might also be useful to consider the variety of tag styles directly when implementing this. Some people like to use #tag, some :tag:. Some also have tag hierarchy like #topic/subtopic or #topic.subtopic. There may be other styles as well (not sure if it is necessary to support all, but at least one or two of the most common ones).

[renerocksai]

At the moment, it is not super clear / complete what I have in mind for tags. Obviously: syntax coloring. That should be easy for normal #tags/including/hierarchy and :tags. I would not go overboard with trying to support too many tag styles that I don't use, where I also don't know how they are used.

Bringing up telescope grep searches for tags is another obvious one: find notes containing tags (considering all tags, specific tag), insert tags, 'follow' tags: find notes containing tag under the cursor.

Tag searches with multiple tags: This is where it gets really interesting: There is option number one, which is telescope. Telescope, I just found out, lets you select multiple items. Standard binding for that is <TAB> in normal mode - which we could override to something else. But a list of tags where you can select multiple tags by itself is not really useful. So after having selected tags in telescope, we would need a second telescope search that actually reveals the matching notes. I am not sure if such a double-telescope search would be useful.

Another question leading to where I want to go: When you select multiple tags, do you want to search for notes containing all of them (and mode) or do you want to search for notes containing either of them (or mode). We could certainly try to have a shortcut that toggles between the two modes in the second telescope finder.

What about finding all notes that contain #project but not #finished?

This is where my other idea comes in: a special buffer type, handily dockable somewhere like the calendar, that can be used for both user input (searches) and output (results). For instance, that buffer could always automatically be populated with actual [[links]] of all backlinks to the currently active note, the one the cursor is in.

Here is a fictional example of it showing details about the current note - which could be templated; another template could show just the backlinks.

# Title
The title of the note

# Tags in this note
 #asdfsadf
 #tag234

# Links in this note
 [[asfdasdfsdf]]
 [[asfdasdfsdf]]
 [[asfdasdfsdf]]
 [[asfdasdfsdf]]

# Back-links to this note
 [[asfdasdfsdf]]
 [[asfdasdfsdf]]

# citekeys in this note
 @renerocksai2021

# notes citing the same sources
 [[The invention of the wheel]]

It could also be populated with [[links to]] results returned by a tag search. Or any search. A bit like a quickfix list but cooler, syntax highlighted, structured.

# Search results for: #that_tag
  [[first matching note]]
  [[another matching note]]
  [[third matching note]]

Once we have that buffer type where results can be placed, why not have a second buffer that can be saved to disk? With that, we could implement features like 'favorites' and 'saved searches'. See also: Saved Searches in Sublimeless_ZK. Consider this:

# My favorites  (totally made up)
[[Projects]]
[[Shopping list]]
[[Goals]]

# Useful searches
most recent notes : [!  {sortby: mtime,    order: desc}
most-referenced notes : =refcounts(min:1, max:1000) {sortby: refcount, order: desc}
unfinished projects: #project & !#finished

Here is a screenshot of what that could look like (from Sublimeless_ZK):

In the above example you see the saved searches at the bottom, and the search results at the top, after the all tags search was executed, resulting in a list of all tags in the search results.

Obviously I would aim for an easier syntax in the form of #tag1 and #tag2 or ( #tag1 & #tag2 ) | #tag3 this time. Implementing functions like refcounts() etc. would definitely be interesting.

I think telescope is super awesome for quickly showing the results of a search, with preview - and then picking one. But a special buffer for search results makes them less transient, more persistent. From there you can yank and paste entire lists of interesting links into your notes... And saved searches can be live-savers (well, time and quality (consistency)) savers for workflows that rely on complex searches that have to be repeated frequently.

So, this is the direction in which I want to take things, loosely related to tags.

😄

[cnshsliu]

WIP

tkbook.mp4

[lambtho12]

THIS LOOKS AWESOME! @renerocksai, did you see this?! 👀

[renerocksai]

THIS LOOKS AWESOME! @renerocksai, did you see this?! 👀

BAM! This goes into a very similar direction compared to what I had / have in mind (and gets pushed into the future all the time)! Really nice work!

[cnshsliu]

Totally your idea and design @renerocksai ! My work is only trying to follow completely with your direction outlined above, just want to implement your brilliant ideas. I am still learning Lua and Neovim API, allow me someday to finish it at some level, Will make a PR soon.

BTW, according to another outline below about the notes map hierarchy:

[[current note]]
 |
 +--- [[child note]]
 |     +--- [[sub child note]]
 +--- [[another child note]]

After the video I posted, I already implement note hierarchy another way, like this:

I only get the one-level parent notes and one-level children notes, and place the current note in the middle to make it a "center note" at this moment.
While move cursor on one of parent or children notes, press CR to open it in the main window.
I also use 'p' to cycle among parent nodes, use 'c' to cycle among children nodes
and "gh" to focus on "current note" wherever the cursor is.
When another note (either parent or child) is displayed in the main window, press 'r', the book window on the right will be refreshed to reflect the change of "current note". and this note become to be the "center node".

This way, we could focus on one "center" note once, and navigate above and below easily. Meanwhile, I also considering the necessarity of showing all notes at any level (like the tree you illustrated) at one time, for me, I keep hundreds of notes, bring them all on the screen with NeoVim may make the navigation harder a little bit. Of course , out of Neovim, especailly with GUI interface, display many many nodes should not be a problem, I also use Node.JS and svelte to program full-stack, maybe someday, I will give it a try with SVG/JS in browser. Ten years ago, I used to keep thought with a PC software named 'The Brain', the centered node is one of it's highlighted features.

My daily language is Chinese, hope my English don't bring vague here. kindly share your thought, and let me know if I am on the right path, thanks a lot.

[cnshsliu]

Still work in progress.

This is what we have at this moment:

1. Note structure (tags, links, todos) and hierarchy (parent and children) navigation
2. Search notes incrementally, by either tag or content
3. Search an AND or OR query (match all words or any of them)
4. Exclude words (match those, but not these)
5. Saved search history, easily pick up again.

am working on Headers outline and navigation

[renerocksai]

Just another thought: bringing up telescope for tag insertion is certainly possible. Inserting tags from there would be easy. If you use multiple selections, all selected tags would be inserted. But we could also leave the picker running after insertion, so one would be able to insert multiple tags one by one.

An additional approach would certainly be: auto completion. I know nothing about how to implement that yet but having a 'completion source' would be super cool for:

• auto completing links (titles)
• auto completing tags
• auto completing headings in special links
• auto completing block ids in special links
• auto completing citekeys from a .bibtex file
• ... more ...?

I know there is a 'null-lsp' thing that lets one feed things like completions into nvim, lsp-style. Maybe that is an option.

[lambtho12]

And you're right, once we start the LSP route, we should be able to do much more than completions: hover-texts might be useful for links, diagnostics (which moved into its own 'subsystem', independent of LSP) could show broken links, syntax coloring, etc. I am sure, many more ideas will come all by themselves smile

LSP could also be used to display note actual title instead of filename in text. This might come handy for people that use abstract titles (e.g. blah blah, see [[202112061246.md]] would display as blah blah, see My Awesome Note Title). I think vim-zettel has something like that implemented (not through LSP though).

[Liquidmantis]

I have to confess that I'm in a Monday morning rush and haven't read all the replies thoroughly, but I'm hoping there's support for bare (octothorpe-less) tags in a tag collection in the yaml metadata. I started using that scheme in Obsidian, but I mostly do my note taking and management in neovim now.

Example:

---
title: My Awesome Note
date: 2021-12-06
tags: [ example, note-taking, foo, bar ]
---

[lambtho12]

I think at some point it will be necessary to enforce only a few styles, otherwise it will become pointlessly complicated and harder to maintain... Usual notations (#tag or :tag) and maybe a tag(s): foo, bar, example line in the yaml formatter are more than likely to cover 95% of the cases.

[renerocksai]

I second that! My aim is to start with the super easy ones like I use (#tag) and keep it flexible: a config setting that defines the style and a lookup table that maps it to a lua function that does the actual parsing.

Regarding LSP things. I recently stumbled across 'extmarks', virtual text that you can place, for instance, at the end of a line. This could be used (without LSP) to display titles next to links. However, the only sane display options I found for those were: at the end of the line and as an (entire) virtual line above or below the mark. This can make it cumbersome when there is [[more]] than [[one]] [[link]] in a single line. Popping up text on hover (LSP style) would probably make more sense. Need to check that out, too...

[renerocksai]

... support for bare (octothorpe-less) tags in a tag collection in the yaml metadata. I started using that scheme in Obsidian, but I mostly do my note taking and management in neovim now.

Example:

---
title: My Awesome Note
date: 2021-12-06
tags: [ example, note-taking, foo, bar ]
---

No worries, we will have you covered. It's now about chopping this all up into pieces for gradual and continuous delivery of (increased) value....

[renerocksai]

We are 1 step further!

I just pushed initial tag syntax support! You can now highlight your #tags, :tags:, and tags: [ tag1, tag2, tag3 ]s!

You can learn more in the README. Check out "Tag notation", and also the install_syntax option, as well as syntax coloring.

Telekasten supports the following tag notations:

1. #tag
2. :tag:
3. bare tags in a tag collection in the yaml metadata:

---
title: My awesome note
date: 2021-12-06
tags: [ example, note-taking, foo, bar ]
---

Tag syntax: I kept this identical to what Obsidian supports. I know how super complex 'tag regexes' can get. So,
spaces are not allowed in tags. To differentiate two or more words in a tag, use one of the following formats:

• camelCase: #noteTaking
• PascalCase: #NoteTaking
• snake_case: #note_taking
• kebab-case: #note-taking

The only symbols allowed are:

• _ : underscore
• - : dash
• / : forward slash

Numbers are allowed in tags, as long as a tag is not purely numeric. For example, #1984 is not a valid tag, but #y1984
is.

Note: For proper highlighting, the install_syntax option is set to true by default. This automatically sets the
syntax of filetype markdown to telekasten, and also registers this syntax with telescope previewers for .md files.

Regarding colors:

" colors for tags: (use magenta or a more gruvbox-friendly purple like below)
hi tkTag ctermfg=175
" tag separator is only relevant in the following syntax:
" tags: [ tag1, tag2, tag3 ] 
hi tkTagSep ctermfg=gray```

---

I learned a lot about syntax, telescope, and plenary:

syntax

What had been bugging me for a while, was that [[links]] and other additional syntax of telekasten would only work when it was not at the beginning of a line. So if I had a link at the beginning of a line, I had to insert at least one space, to get proper highlighting. When I implemented the tags: [] syntax, this became super apparent.

After a while, I figured out: if I load the telekasten syntax a second time (:source telekasten/syntax/markdown.vim), it would work! Without inserted spaces.

So eventually, I worked it out: To avoid mixing markdown syntaxes in strange orders (depending on what other plugins one uses), I had to define a telekasten syntax. If I :set syntax=telekasten, the syntax works as expected. To get normal markdown syntax, too, I source the markdown syntax at the beginning of the telekasten syntax.

With the new parameter install_syntax set to true (default), telekasten issues an autocmd: autocmd filetype markdown set syntax=telekasten automatically, at Setup(). But this parameter does more...

pre-viewing the new syntax

Telescope does not use conventional vim methods to figure out the file type of a file. That would be too slow for the
previewers. Instead, it uses plenary.filetype. Unfortunately, plenary.filetype cannot know that we need telekasten
syntax for the markdown filetype. However, a mechanism exists in plenary.filetype for injecting a lua function that
can define syntaxes for file types, by filename extension.

Its only drawback, I thought, was: that it looks into ~/.config/nvim/data/plenary/filetypes/ for lua files. At least
that's what I found in the help file. I tried it and it worked. I could set the new syntax for .md files.

But then I looked into the code: it uses vim.api.nvim_get_runtime_file with the path data/plenary/filetypes/...lua.
This is awesome! Because that nvim function tries ALL runtime paths! And plugins get their own runtime
paths! So by just adding the injection code into the new subdir telekasten.nvim/data/plenary/filetypes/,
we can add the new mapping that forces telescope previewers to use our own syntax!

The only drawback is: The file supplied to plenary is fixed. This fixes the markdown file extension to .md. I could add more, like .txt and so on. I also thought about generating this file automatically. After all, at setup() time the extension is known. But since it needs to live inside the plugin, I am not sure whether that would confuse plugin managers - unclean git checkouts...

So, right now I don't know what the best option is: put all expected file extensions into the new file - and a note in the README where to change it (which could lead to trouble after plugin updates by the plugin manager: overwriting or refusing to update). Or - I could try if anyone has a problem when the file gets auto-generated with only the configured extension. I lean strongly towards the latter.

[Liquidmantis]

I need to actually start working, but just for an update I did remove all plugins other than telekasten, telescope, and plenary. Even with this bare (and terrifying) config the various highlights aren't working for me.

[renerocksai]

This is so weird! I totally believe that you're setting the highlights correctly.

I use tmux inside of kitty on Ubuntu. I don't want to believe this is a mac issue. I usually alias tmux to tmux -2 to enable 256 colors but that should not be the problem because you don't get the highlighst even without tmux.

I take it, your :set syntax? is telekasten, too?

One thing you could try to get more insights is going to .config/nvim/bundle/telekasten.nvim/syntax/telekasten.vim (adjust to your plugin paths) and remove the line 2 that says runtime! syntax/markdown.vim. When you do this, only the few telekasten specific syntax elements will be active: tkTag, tkBrackets, tkLink, tkTagSep - without any other markdown syntax getting in the way.

If that works, then at least we know where to investigate next.

[renerocksai]

🤦 AAAAAH, I just got an idea!

You probably have termguicolors set

This is what I start with (no termguicolors)

I enable termguicolors

And the explanation is in :h termguicolors:

The solution

My examples never set the guifg=....

Try:

hi tkTag ctermfg=175 guifg=#d3869B

This gives me the tags back:

So the solution is to update the documentation so it includes guifg and guibg colors as well as ctermfg and ctermbg.

[renerocksai]

And to save you some trouble, here is my new color config:

hi tklink ctermfg=72 guifg=#689d6a cterm=bold,underline gui=bold,underline
hi tkBrackets ctermfg=gray guifg=gray

" real yellow
hi tkHighlight ctermbg=yellow ctermfg=darkred cterm=bold guibg=yellow guifg=darkred gui=bold
" gruvbox
"hi tkHighlight ctermbg=214 ctermfg=124 cterm=bold guibg=#fabd2f guifg=#9d0006 gui=bold

hi link CalNavi CalRuler
hi tkTagSep ctermfg=gray guifg=gray
hi tkTag ctermfg=175 guifg=#d3869B

[Liquidmantis]

And there was much rejoicing! Thanks so much for putting in all the effort. That does indeed solve the issue.

[renerocksai]

Check out :Telekasten show_tags / :lua require('telekasten').show_tags()

Config option: show_tags_theme

Supports <c-i> (insert tag) and <c-y> (yank tag) and <cr> (search notes for selected tag)

BTW it shows how many matches there are in your notes, per tag:

[renerocksai]

Good point with the anti-pattern! But even with a concept like a zettelkasten: you do you! Dogma bad! 😄

If you have a good use-case, then go for it, even if it might seem counter to someone else's philosophy of what a zettelkasten should be. After all, we're mixing it with a calendar here! 😄

Regarding Obsidian: I have it installed, too. I used to be quite fond of it. But eventually, its vim-emulation left me wanting for more. I knew I wanted a '(n)vim' solution. With Telescope coming along I figured, hey, here is a perfect TUI already... I digress.

My Obsidian handles #tags like described in its help. It appears, #tags are first-class citizens. When you look up tags, you find #this-notation there. No need for quoting them (only if you put #tags in the yaml front matter). So I don't see a need for using yaml tags. I find them too limiting anyway. What I do instead is: comments!

---
title: My awesome note
---

<!-- tags: #zettelkasten #tags #telekasten -->

# Heading 1
blah here we mention another #tag 

I found this to be a really good and easy and flexible solution. It won't allow processing front matter by some yaml tool - but I don't care about that anyway.

Going

---
tags: [ '#tag1', '#tag2' ] 
---

is something worth considering, though.

I haven't used Obsidian in weeks now. I don't miss it. When I started out with telekasten.nvim I thought of it more as a compagnion, now it's a replacement for me. With 'markdown preview' (see recommended plugins in the README) I even get some sort of <ctrl>+<e> experience when I need to see a note with inline images.

[Liquidmantis]

Sorry, we crossed replies and I missed this one.

On migrating away from Obsidian, I agree! I hardly fire it up anymore other than to use it to find orphaned (unlinked) documents and to use the graph visualization. This plugin is moving toward the features to make Obsidian totally unnecessary.

I really hope this doesn't sound rude, but the tag in comments like that feels a little inelegant. It's like duplicating metadata blocks... although... I think putting it as a footer would work well for me and not offend my delicate sensibilities. 😆 Then you have to look for the clutter rather than having it top dead center. Similarly, I really like how Obsidian can fold the yaml block. I need to see if I can implement something like that with vim folds.

I'm totally open to any solutions for getting a visual map of note relationships in neovim or at least the terminal, if you have any.

[renerocksai]

These discussion threads - I wish there was an easier way to find updated threads.

I like my comments! When I started out, I couldn't stand yaml mixed into my markdown. But HTML was OK 😄 . I decided to do what works for me and I don't see myself ever processing my notes through some tool that pulls out the yaml and does something with it.

Now, visualizing note relations is worth a discussion on its own. One day I will experiment with using dot to visualize a graph of notes connected to the current one and displaying it with Überzug, just like the preview buffers for images. I guess, the resolution of a text buffer is not really helpful here, hence the graphics. Obviously, that wouldn't be interactive. So maybe one can overlay interactive HTML + js, rendered in some web view, overlayed.... These are just wild and speculative ideas. Maybe for now, the good old text tree is OK:

[[current note]]
 |
 +--- [[child note]]
 |     +--- [[sub child note]]
 +--- [[another child note]]
       

Does image preview work on macOS?

[Liquidmantis]

These discussion threads - I wish there was an easier way to find updated threads.

Yes! This is the most I've used GH Discussions, and I thought the threaded replies would be great for keeping in-thread topics threaded, but it's easy to miss replies. I try not to use @ tags in direct replies, but maybe that is the expectation for notifications.

I like my comments! When I started out, I couldn't stand yaml mixed into my markdown. But HTML was OK 😄 . I decided to do what works for me and I don't see myself ever processing my notes through some tool that pulls out the yaml and does something with it.

I hear that. I usually hate and try to avoid mixing languages. But after Obsidian introduced me to the idea of yaml frontmatter (thanks to @lambtho12 for reminding me of the name) in markdown I grudgingly accepted it as at least it's a supported mix.

Does image preview work on macOS?

There are tools for graphviz, but there's no direct replacement for ueberzug (or Überzug for you fancy Germans) that I've found. This week I saw a mention of using Ranger in the terminal as a possible solution. That's one my exploration list for this weekend.

But you're right, I bet someone has a node.js tool for generating the maps... At least until you get the bug to create something purpose-made. 😉

[renerocksai]

Well, Überzug is only required if you want to render an overlay over an nvim buffer. A cool alternative could be: some
node.js tool that draws interactive maps. I am not sure but there is a tool called "foam" that uses vscode to do
roam-like stuff. Maybe it comes with a graph display. One could get inspired by that. A counterpart in the lua plugin
that notifies the web thingy when the cursor changes to a different note would be cool. Clicking on a note in the graph
could signal the plugin to open the selected note in the current window. Markdown-preview syncs between nvim and its web
view of a note....

There's just too many ideas. One step at a time. ATM my focus is on getting the backlog smaller... But feel free to try out things! Maybe something cool will come from it!!!!

[Liquidmantis]

So far the tag searching isn't working on my machine. I have rg installed, but I get this error:

E5108: Error executing lua .../site/pack/packer/start/plenary.nvim/lua/plenary/job.lua:493: 'rg --vimgrep -o (^|\s)tags:\s*\[([a-zA-Z]+[a-z
A-Z0-9/\-_]*(,\s)*)*] -- /Users/liquid/Dropbox/documents/notes' was unable to complete in 5000 ms                                          
stack traceback:                                                                                                                           
        [C]: in function 'error'                                                                                                           
        .../site/pack/packer/start/plenary.nvim/lua/plenary/job.lua:493: in function 'wait'                                                
        .../site/pack/packer/start/plenary.nvim/lua/plenary/job.lua:447: in function 'sync'                                                
        ...k/packer/start/telekasten.nvim/lua/taglinks/tagutils.lua:119: in function 'do_find_all_tags'                                    
        ...ite/pack/packer/start/telekasten.nvim/lua/telekasten.lua:1571: in function 'show_tags'                                          
        [string ":lua"]:1: in main chunk  

[renerocksai]

Okay, so no errors when using #tag notation. However, the search ignores the yaml metadata tag collection when they're quoted. Unquoted is found in the search, however.

---
tags: [ #thisTagWillBeFound ]
---

---
tags: [ "#thisTagIsNotFound" ]
---

In that case, the tag block isn't compatible with Obsidian, and is apparently invalid yaml.

Ooooh! I need to add support for those quotes! I forgot about: tags need whitespace or else the would be found in https://some-link.com/page#anchor.

I'm getting onto that.

[lambtho12]

Personally, I use a bit of both worlds.. I have regular inline tags in the yaml frontmatter and it all works great. No need to do some fancy stuff... 😄

---
tags: #tag1 #tag2 #tag3
---

[renerocksai]

BTW 6429239 just fixed the #tags surrounded by quotes - which one needs if the yaml needs to be valid, for other tools etc.

[Liquidmantis]

BTW 6429239 just fixed the #tags surrounded by quotes - which one needs if the yaml needs to be valid, for other tools etc.

This fixes all my use cases (so far... 😆 )! I now have telekasten configured to use #tag notation, which works with tags in the yaml frontmatter and inline tags. Searching returns them both, and it still maintains full compatibility with Obsidian. Huzzah!

[renerocksai]

BTW you know you could have gotten the same thing with comments. Put just below the yaml header... 😂 I'm only kidding.