Move doc from README to wiki?

[lambtho12]

The README is becoming a little bit longer every week (currently +850 lines), and it is not so easy to navigate. I am not super familiar with the github wiki stuff, but it seems to be a bit more appropriate to host such a detailed documentation and make it easily navigable.

However, I am not sure if it would be possible to have it accessible to everyone for documentation PR and stuff.

What do you think of it?

[renerocksai]

Hi @lambtho12, I am all for improving the documentation. If that means, cutting the README and linking to parts in the Wiki, I am all for it.

However, I am not sure about accessibility - don't know how that works. If it's kind of "pull request" based - that could work, couldn't it?

[lambtho12]

Nope... There is no web interface and such to propose and discuss PRs, issues, etc for wiki repos. This is complete nonsense, people have been complaining about that for years on Github or Gitlab alike. It seems that they both use Gollum as a backend for that and no one is willing to actually implement that feature.

After hours of searching for a what to do it, it seems that there are only three options

1. Someone does the change locally, push to their fork of the repo, open an issue ticket to tell you that they made changes, you pull these locally, review locally and merge locally to the official repo before pushing it (what we did in [Wiki additions] #77)
2. Abandon the wiki idea, create a completely separate repo for the doc and use a deployment pipeline with a static site builder (readthedocs, hugo,...) to build the documentation after each push automatically and publish it on githubpages for instance.
3. The Tor Project uses a separate repo that is automatically mirrored with the wiki, so the changes to the repo are pushed to the wiki (see here), this requires setting up authentication tokens, etc. And they are on gitlab, so I do no know exactly if it is even possible on github.

Other possibilities (but without proper reviewing process)

• Opening the wiki publicly so anyone logged in github can change it (can lead to vandalism, etc). Not ideal, IMO
• Elevating trusted people to the collaborator rank on this project and allowing collaborators to edit the wiki. (Still no public trace of the changes)

[renerocksai]

What you you think of that: https://stackoverflow.com/a/58692638

They use a doc/ folder and some GitHub action to generate the wiki from it. Apparently, the wiki view seems to be nicer.

So we could author wiki pages locally in the doc folder, use PRs, etc.

It's not exactly the same as a real wiki experience, though.

I am also wondering: if we have to do it locally anyway, why not use telekasten for the wiki? That opens up another can of worms: multiple note archives.

What I can try is: creating a link from telekasten's not yet existing wikidocs folder into telekasten home. That would work, [[links]] should also work - but this might be sub-optimal if I have identical links in both my normal notes and the wiki. Both will be displayed so it should not be too bad but still.

Or another idea: a telekasten switch_home function that re-runs setup, just with a different home... Or, even less error prone: a toggle function behind some key bind that toggles between calling setup for my home and setup for the wiki.

This, combined with a CI action that creates a GitHub wiki from the wikidocs folder, could be .. OK.

Not as accessible as a PR-enabled wiki but not bad, either.

[lambtho12]

What you you think of that: https://stackoverflow.com/a/58692638

They use a doc/ folder and some GitHub action to generate the wiki from it. Apparently, the wiki view seems to be nicer.

Mmmmm, yeah this is basically what I proposed in step 2, except that they do not use a static site builder, but push automatically the changes in the documentation folder to the wiki repo. That allows them to benefit from the github wiki feature and avoid having to go through setting up a proper template and github pages. Nice idea, keeps the doc somewhat simple, and more importantly, it keeps it within the repo (easier to do a single PR with updated doc and code).

Regarding the nicer wiki view, it is because the customized the sidebar, footer, etc. I am planning to do that as well when all pages are setup.

---

Why not use Telekasten for the wiki?

Good point, but there will be a few more issues than just dual home for telekasten. I am not sure that Github wiki will recognize [[links]] for instance, so we should allow markdown format as well (which if I recall was ruled out, at least for now). That would require some testing.

telekasten switch_home function

As someone who likes stuff neatly organized, I definitely see a point in doing that. Maybe even more versatile would be telekasten.set_currentcwd_as_home so we could open any file anywhere and set its path a the home of a ZK.

[renerocksai]

I just checked here: GitHub wikis do support [[mediawiki style links]].

OK, I like your suggestion. Initially, I was worried about all the 'subdirs' for dailies and weeklies, images. But: if we just switch the home dir while still being able to use the journal features (dailies), that could be more a feature than a bug. The image_dir I am not sure...

The most flexible thing to do would still be to re-run setup. So, for example, if you use .zmd as extension in your normal home and need to use .md in the wiki - that's no problem with setup.

However, switching home dir on the fly is probably a more common use-case. I would not like cd-ing into the wiki for that - like, leaving nvim or whatever. So a telekasten.switch_home() could accept opts or at least a parameter. telekasten.switch_home(vim.fn.getcwd()) in your case, telekasten.switch_home('~/....../wikidocs') in my case. We could argue to make the result of getcwd() the default in case no parameter is passed in or if it is nil.

[lambtho12]

Yep good point.

The global_dir_check function would maybe need a bit of a rework then (I am currently not quite fan of how it work anyway). I do not think it should check if all repos are setup everytime we try to use a function, especially if we are not working on our main zettel.
What bothers me with it currently is that it will force me to decrypt my journal directory even if I just want to paste an image in some regular note for instance. And there is no way around that.

[renerocksai]

Another thought: If we strip the README...

Lots of stuff will move into the Wiki. Fair enough. But then the correlation between README and help file gets weaker.

Where would we recommend documenting new features? In the README? In the Wiki?

Maybe if we just give it a go we'll find out anyway 😄

[lambtho12]

correlation between README and help file gets weaker

Is that a problem? I am not the most experienced software developer, but to me the README should explain what is the project, how to install it and some basic configuration and usage. I would put the rest goes into the full-blown documentation and help file. See for instance the vimtex plugin. I believe that the majority of people just want to quickly add the stuff, copy/paste a sane default config and get on using it. No need to scare them with README of a million line that goes over every detail.

Where would we recommend documenting new features? In the README? In the Wiki?

The complete documentation should go into doc/telecasten.txt and wiki.

To notify users of new features, we could:

• Tag major additions and add a CHANGELOG.md
• Add a "what's new" in the README where we would keep a short list of latest major addition
• Both

[renerocksai]

These are insanely good suggestions! 👍

Re problem: no, it's not a problem per se. It was super easy to point people to 2 files. But as you pointed out, different does not necessarily mean problematic.

A changelog seems OK - we'd add that to the PR template. Regarding the changelog and the README: I think, the Changelog can get quite lengthy over time. Scrolling past dozens of lines of meaningless changelog entries is probably also against the spirit of a README.

As a compromise, we could always take the 5 most recent changelog entries and put them in the README, with a link to the Changelog at the bottom. But this appears to be high-maintenance. So, I would opt to :

• tag major additions (including PRs)
• add a CHANGELOG.md (kind of the done list in the backlog.md)
• add a what's new to the README with a short list of latest major additions, no need to update this with every changelog entry
• announce major additions in the discussions?

What do you think?

[lambtho12]

I think, the Changelog can get quite lengthy over time. Scrolling past dozens of lines of meaningless changelog entries is probably also against the spirit of a README.

Agree. Should be a separate file CHANGELOG.md with a simple link to it from README.

• tag major additions (including PRs)
  - add a CHANGELOG.md (kind of the done list in the backlog.md)
  - add a what's new to the README with a short list of latest major additions, no need to update this with every changelog entry
  - announce major additions in the discussions?

Agree with all of that! Add a link to CHANGELOG.md there to send user to the full list easily.

[lambtho12]

@renerocksai I have more-or-less finished the Introduction section of the wiki. See it on my fork. I think it makes it clearer and simpler to navigate. So far I did not change fundamentally anything besides reorganizing stuff, improving the layout and removing unrelated comments.

I will continue working on that this evening and the rest of the week (depending on my availability). Then I will make an issue request and all that stuff when everything will be done.

[renerocksai]

Wow! I am speechless! That sounds great! Will have a closer look later.

👍👍👍

[renerocksai]

Double wow!!! OMG! You have reality put a lot if thought and work into this wiki! I like it a lot!!!

I am on my mobile now but will pull this as soon as I get a chance!

(Edit: unless you want me to wait?)

[lambtho12]

Nope, it is all good now. I added an issue for that PR (#80) so it can be tracked/referenced. So merge whenever you want!

I think the next step is to declutter the README quite a bit, properly link to the wiki everywhere possible and implement the CHANGELOG and tags as we discussed hereabove.

I do not know if I am comfortable doing that. The README is the first page people see when they open the repo. As it is your project, it should be you to decide what you want there IMO.

[lambtho12]

BTW, I think you can remove the subdirectory pointing to the wiki from the project. I do not think it is really useful...

[renerocksai]

@lambtho12 , thanks for your support and all the great work. At the moment, I hardly have any time for Telekasten.nvim - and this is likely to continue for the next two months. But if you like - I have added you as a collaborator to this repository. As you've proven yourself already 😄, please feel free to merge PRs etc, as you see fit. Let's make this more a community effort - without too many gatekeepers (bottlenecks like me at the moment) - if you like.

[lambtho12]

Thank you for this offer. I gladly accept the task 😄
I am fairly busy as well for the moment, but I will try to cram a few hours here and there to continue the development.

[lambtho12]

I FOUND A WAY!
I am not sure if this can be done with github workflows, but using Gitlab, I managed to setup a pipeline so every time I modify something in the docs folder, it clones the wiki repo, removes everything in it, copy back the docs folder in it and commits & push to the wiki. Here is my configuration.
@renerocksai , when you have some more time to work on TK, maybe you can try to adapt this to github so we can have a better doc system.

---
push-wiki:
  # Following methodology in https://gitlab.com/gitlab-org/gitlab/-/issues/16261
  image: bitnami/git
  script:
    - export WIKI_URL="${CI_SERVER_PROTOCOL}://project_${CI_PROJECT_ID}_bot:${BOT_TOKEN}@${CI_SERVER_HOST}:${CI_SERVER_PORT}/${CI_PROJECT_PATH}.wiki.git"

    # Remove old clone and re-clone the project's wiki
    - rm -rf "/tmp/${CI_PROJECT_NAME}.wiki"
    - git clone "${WIKI_URL}" "/tmp/${CI_PROJECT_NAME}.wiki"

    # remove everyting from the wiki and copy docs/ content to it
    - rm -rf "/tmp/${CI_PROJECT_NAME}.wiki/*"
    - cp -r docs/* "/tmp/${CI_PROJECT_NAME}.wiki/"

    # enter the cloned wiki
    - cd "/tmp/${CI_PROJECT_NAME}.wiki"

    # edit the links to work with wiki
    - sed -i -E "s|(\(docs/)|\(|g" *.md
    - sed -i -E "s|(.md\))|\)|g" *.md
    - sed -i -E "s|(.md\#)|\#|g" *.md

    # set committer info
    - git config user.name "$GITLAB_USER_NAME"
    - git config user.email "$GITLAB_USER_EMAIL"

    # commit everything and push to master
    - git add --all
    - git commit -m "Pushed docs with CI"
    - git push origin "HEAD:master"

  only:
    refs:
      - master
    changes:
      - docs/*

Note that I use some sed command at some point to reformat the links. This is because the links should be formatted differently if you browse the wiki or if you browse md files in the he docs folder.

[lambtho12]

https://github.com/OrlovM/Wiki-Action