Writing A Fiction Novel (TabbyML for writers)?

[carcinocron]

Bear with me, I have a pretty tenuous grasp at what TabbyML is, aside from "Self-hosted Github Copilot".

Assuming a writer has a folder of markdown files roughly similar enough to:

book
└── book1 - The Friends We Made Along The Way.md
    ├── chapter_1.md
    ├── chapter_2.md
    └── chapter_3.md
wiki
├── loose-ends.md
├── outline
│   └── book1.md
├── people
│   ├── Salazar (captain).md
│   ├── officer_tucker.md
│   ├── sally.md
├── power
│   ├── fireballs.md
│   ├── invisibility.md
│   └── mind_eating.md
├── species
│   ├── humans.md
│   ├── klingons.md
│   └── fairy.md
├── TODO.md
├── themes.md
└── quote_ideas.md

If the author were willing to switch from Word.exe or Obsidian to VSCode and install TabbyML,

• how useful would this this extension be?
• what model(s) would be best to try?
• Are there currently any alternative free and/or open source projects intended for the use-case I have presented?

[carcinocron]

I gave it a test drive using the default model on MacOS (TabbyML/StarCoder-1B):

I think it doesn't work (no autocomplete offered) if the current file is too large. I couldn't find where to check for error messages.

For the times it did work, maybe a "reroll" hotkey would be nice?

Technically none of the autocompletes were usable (nonsense in the context of the paragraphs I didn't includes in the screenshots, repeating itself), most likely because the model I was using is small and trained for a completely different purpose, but I think it shows potential.