0.20.0

[MohamedBassem]

Welcome to the 0.20.0 release of Hoarder! This release adds internationalization support, new markdown editor, mobile app improvements, broken link management and more! As usual, we're welcoming our new contributors: @Tukks, @vhsdream, @sweepies, @PeterDaveHello, @wuast94 and @eltociear!

If you're enjoying Hoarder, you can consider supporting it here ☕️ or via github here.

New Features

• Hoarder's UI now comes in multiple languages!
  • You can change the language of the UI now to English, German, French, Swedish, Simplified and Traditional Chinese.
  • If you want to see your language there or want to improve some existing translation, you can do that via weblate here.
• New experimental WYSIWYG markdown editor for notes by @Tukks!
  • You'll now get a preview of the notes you're writing as you're writing them!
• A bunch of mobile app improvements:
  • You can now manage tags from the app. Super overdue, I know.
  • We have a new cleaner "info" page with the ability to edit title/tags/lists all in one page.
  • The mobile app now looks slightly more native. We've got rid of the dedicated search tag, switched to native modals, and used native page headers when possible.
  • Note: The apps are yet to be approved by apple & google.
• Broken Link management:
  • Hoarder now stores the status codes it got when crawling websites which will allow you to identify and decide what to do about broken links.
  • This is now mainly useful for people doing large imports into hoarder, but in the future, we'll use the same mechanism to support automatic link liveness healthchecks.
• Chrome as an optional dependency (aka minimal installation):
  • Chrome is no longer a hard dependency. This is useful for people running hoarder in resource constrained environments.
  • If you don't specify a BROWSER_WEB_URL, hoarder will fallback to a plain HTTP requests to the websites you add instead of crawling them with chrome.
  • Without chrome, you'll lose the ability to get screenshots from the websites you crawl, and websites that heavily rely on javascript won't work properly.
  • With chrome and meilisearch being optional dependencies, you can now get a single-container minimal installation of hoarder by sacrificing some of the features.
• New installation methods:
  • Debian / Ubuntu one click installation script (docs) by @vhsdream
  • If you're a proxmox user, you can also get a one click LXC container running hoarder using proxmox's helper scripts again thanks to @vhsdream.
  • PikaPod (docs) are now offering paid hoarder hosting (~$3/month) for the less tech savvy folks out there. Disclaimer: they're sharing some of their revenue from Hoarder with us.
• Tags placeholder in AI prompt customization:
  • You can use the placeholders $tags, $aiTags, $userTags in the prompt. These placeholders will be replaced with all tags, ai generated tags or human created tags when automatic tagging is performed thanks to @kamtschatka.
  • With that, you can use this to instruct the LLM to only use existing tags (instead of coming up with new ones). Be careful though if you have a lot of tags as it might increase the cost of the tagging significantly.
  • This mechanism is just a temporary solution. We're working on a mechanism to support this natively, in a much cheaper way.
• New community projects documentation page (here):
  • With the release of the REST API in the last version, we started seeing some community projects pop up. So we're starting a new documentation page to list them out for visibility.
  • Hoarder now has a raycast extension and an alfred workflow thanks to @foru17 and @yinan-c. Go install them now!
  • If you develop a new extension for hoarder, let us know as we'd love to feature it in that page!
• Others:
  • We've added the ability to show admin warnings in the admin panels for misconfigurations or future migrations.

UX Improvements

• We're now trimming email addresses out of whitespaces during signin thanks to @sweepies!

Fixes

• AI summarization sometimes included the LLM's response to the prompt (e.g. "Ok, I'll summarize ..."). We're now instructing the model to only respond with the summary.
• Fix broken homepage when hitting a bookmark with an invalid image URL.
• Multiple worker reliability improvements specially in countries with restricted internet.
  • The worker container doesn't need to download pnpm on startup anymore as it's now bundled in the container.
  • The crawler's adblock list download now has a timeout and can be completely bypassed with CRAWLER_ENABLE_ADBLOCKER=false.
  • The worker was sometimes hanging in the screenshot call. This now has a timeout as well.

Screenshots

The new Markdown Editor

Broken Link Management

Mobile App Improvements

New options at the bottom of the bookmark preview page

Tag Management

The new info page

Admin Notices

Raycast extension and Alfred workflows

Those screenshots are from the authors of the extensions.

Raycast extension by @foru17

Alfred workflow by @yinan-c

Upgrading

To upgrade:

• If you're using HOARDER_VERSION=release, run docker compose pull && docker compose up -d.
• If you're pinning it to a specific version, upgrade the version and then run docker compose pull && docker compose up -d.

All Commits

• i18n: Enable the Swedish translation - @MohamedBassem in e3b8cda
• i18n: Merge translations from weblate - @MohamedBassem in ab1f49b
• feature: Add an admin notice about the usage of the legacy container images - @MohamedBassem in 4bfb3b4
• release(mobile): Bump mobile version to 1.6.7-4 - @MohamedBassem in 2245887
• deps: Upgrade nextjs to 14.2.15 - @MohamedBassem in 7c05bc8
• deps: Upgrade prettier to 3.4.2 - @MohamedBassem in 1aa7639
• docs: Add the community projects page to docs - @MohamedBassem in 973d552
• feature: WYSIWYG markdown for notes. Fixes Improve note support : WYSIWYG markdown #701 (#701 Improve note support : WYSIWYG markdown #715) - @Tukks in 40c5262
• Translated using Weblate (German) - @murray3k in 14f8937
• Translated using Weblate (Dutch) - Guido (via weblate) in 17fdd93
• Added translation using Weblate (Dutch) - Guido (via weblate) in 25ebd8e
• Translated using Weblate (Swedish) - Marcus skoding (via weblate) in 20968ec
• Added translation using Weblate (Swedish) - Marcus skoding (via weblate) in 883087b
• fix: trim whitespace from email on signin page (Trim whitespace from email on signin page #714) - @sweepies in 16f2ce3
• fix(mobile): useLayoutEffect for navigator changes - @MohamedBassem in 6fba9d1
• ui(mobile): Use native search bar for manage tags page - @MohamedBassem in deadcb9
• fix(mobule): Fix keyboard issues in the info screen - @MohamedBassem in 6055f50
• ui(mobile): Make page titles more native - @MohamedBassem in f4b2bac
• feature: Store crawling status code and allow users to find broken links. Fixes Improve handling of failed crawling jobs #169 - @MohamedBassem in 705d539
• docs: Add Linux install script and documentation (Add Linux install script and documentation #707) - @vhsdream in a7b1386
• feature(workers): Allow running hoarder without chrome as a hard dependency. Fixes Feature: Minimal install, no AI, no crawler. Hoarding only! #650 - @MohamedBassem in 1810100
• fix(mobile): Use keyboard avoiding view in the info page - @MohamedBassem in 5a49691
• fix(extension): Fix dev build. Fixes Cannot connect to the Vite Dev Server on http://localhost:5173 #670 - @MohamedBassem in 1a2b600
• fix: Instruct the model to only respond with the summary when summarizing content - @MohamedBassem in b9f1a59
• fix(workers): Add spaces in tag placeholders for better tokenization - @MohamedBassem in 036554a
• feature: Add support for tag placeholders in custom prompts. [Feature request] Force AI to use existing tags (instead of creating them) #111 ([Feature request] Force AI to use existing tags (instead of creating them) #612) - @kamtschatka in fdf28ae
• release(mobile): Bump mobile version to 1.6.7-2 - @MohamedBassem in 82af305
• docs: Fix docs build - @MohamedBassem in 6392cca
• deps(mobile): Upgrade to Expo 52 (deps(mobile): Upgrade to Expo 52 #692) - @MohamedBassem in fd4a996
• release(mobile): Bump mobile version to 1.6.7-1 - @MohamedBassem in d32457e
• feature(mobile): Add support for creating tags from the manage tags page - @MohamedBassem in 2e5cd59
• fix(mobile): Dont dismiss keyboard on tag selection - @MohamedBassem in dd80f91
• release(mobile): Bump mobile version to 1.6.7 - @MohamedBassem in bf80fa2
• feat(mobile): Add support for managing tags from mobile - @MohamedBassem in 2745330
• feat(mobile): Allow editing titles and a cleaner info page - @MohamedBassem in fbb2644
• ui(mobile): Replace bottom sheet with native screens (ui(mobile): Replace bottom sheet with native screens #690) - @MohamedBassem in 5522e20
• readme: Add weblate translations to readme - @MohamedBassem in 4bb7487
• ui(mobile): Remove the dedicated search page and add a search bar in home - @MohamedBassem in cc84e01
• docs: Add pikapods.com as installation option (Add pikapods.com as install option #676) - @m3nu in df19d39
• i18n: Add zh-TW Traditional Chinese locale (Add zh-TW Traditional Chinese locale #689) - @PeterDaveHello in dcba3c4
• fix(workers): Set a timeout on the screenshot call and completely skip it if screenshotting is disabled - @MohamedBassem in 6ccdbe5
• fix(workers): Don't block connection to chrome when failing to download adblock list. Failed to connect to the browser instance, will retry in 5 secs #674 - @MohamedBassem in 378ad9b
• chore(workers): Add extra logging for browser connection errors - @MohamedBassem in 393d097
• readme: Add github sponsor - @MohamedBassem in 5533632
• docker: Preinstall pnpm for the worker job to prevent downloading it on startup. "no such table: tasks" when adding a new item #666 - @MohamedBassem in ae78ef5
• fix: Fix homepage not loading when hitting adding invalid URL during crawling. Fixes Bookmarks on Homepage not showing up #626 - @MohamedBassem in dd20fb7
• feature: Allow setting bookmark metadata during creation - @MohamedBassem in 82cd3bb
• docs: fix typo in inference.ts (docs: update inference.ts #640) - @eltociear in 64e759a
• translations: Add german, french and chinese LLM translations - @MohamedBassem in 0d6af2f
• feature: Add i18n support. Fixes feature request: i18n support  #57 (feature: Add i18n support. Fixes #57 #635) - @MohamedBassem in 4354ee7
• docker: Upgrade meilisearch to v1.11.1 (upgrade meilisearch to v1.11.1 #629) - @wuast94 in 9f2c7be
• fix: Fix SummarizeWithAI button to use an ActionButton - @MohamedBassem in d73f001
• docs: Release the 0.19 docs - @MohamedBassem in bff02d6
• readme: Add RSS, REST API and video archival to README - @MohamedBassem in 6ccaec9

---

This discussion was created from the release 0.20.0.

[KristofStroobants]

Hi, I'm a Windows server admin, starting to dabble more and more into a home lab.
I run proxmox on an old surface with Home Assistant being my main reason. Playing around I discovered Hoarder.
It's the thing I never knew I needed. It is amazing, so first off, thanks for creating it.

Currently it is running in docker. But to be fair, I don't like docker as much and prefer LXC. Mostly due to the way docker works with its network assignment. I know you can use macvlan etc but I just don't like it that much.

My question is, with the LXC, how do you update your version from say 0.19 to 0.20 (or whatever case it may be) ?

[vhsdream]

Hi @KristofStroobants - welcome to the wonderful world of self-hosting! For the Hoarder LXC Proxmox Helper Script, it's as simple as typing update into the LXC console window and hitting Return.

I've just tested it now in fact, and it works perfectly:

[Cantello]

Now that different languages for the UI are available, I wanted to tackle the problem that tags that are automatically generated are always one language. I know that the inference language can be changed via INFERENCE_LANG but can the prompt be changed in a way that for links in language A, tags in the same language will be generated?

[MohamedBassem]

@Cantello That sounds like something that can be done with the prompt customization. Have you tried adding the following to the custom prompt:

Generated tags MUST be in the same language as the bookmark being tagged

[Cantello]

That worked, sometimes a solution is really simple, thanks!

[kamtschatka]

actually there is no validation in the inference language so if you put "the same language as the input" into INFERENCE_LANG, this should already work and prevent any confusion in the AI, as the part
"- The tags language must be in ${lang}." is always added automatically.
If you add "the same language as the input", this should then simply be part of the prompt.
Your mileage may vary, but ideally the whole prompt would be configurable, as there have been many requests to do that before.

[MohamedBassem]

@kamtschatka Yeah, I thought about the free form of inference lag as well, but decided not to suggest it because maybe it works with the current prompts, but might not work with future prompts where we reuse this variables. That's why I think the new rule should be good enough.