add script to generate docs _sidebar.md based on the /docs content

[Payback159]

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design

/kind documentation

/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:

I have noticed in past pull requests that if the PR creator does not take into account that his documentation is also linked in the _sidebar.md then it can be found in the search of kubespray.io but not in the menu shown on the left.

The script should make it easier to automatically generate the _sidebar.md from the content of the /docs folder in the future.

In further steps it could perhaps even be started as part of the pipeline to keep the kubespray website up to date.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

I have moved the existing documents in the docs folder to the various subfolders using the existing sidebar structure. I use the subfolders to create the "root structure" of the sidebar.

For the Markdown files that were located directly in /docs, I tried my best to divide them into the right categories.

The _sidebar.md file in this PR has already been created with this script, as an example of what the result would look like.

Does this PR introduce a user-facing change?:

BREAKING CHANGE: This script is introduced to facilitate living documentation and its administration. This leads to a restructuring in the documentation at https://kubespray.io/#/ to simplify the automatic creation of links, as the structure in the sidebar changes.

[k8s-ci-robot]

Hi @Payback159. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[yankay]

/ok-to-test

[yankay]

Thanks @Payback159
Amazoning work! 👍

It will make some links broken in https://kubespray.io/#/ , so it's a broken change.
We should add it to the notes :-)

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Payback159, yankay

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [yankay]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Payback159]

Hello @yankay,

thanks for the feedback. I have adjusted the notes to make it more obvious that there is a break in the documentation here.
I have also fixed the merge conflict with master branch.

Let's see what the CI has to say :D

[VannTen]

The new structure looks OK to me (though I always grep through the docs, so it's not like I would notice ... ^). One question though: docs/CNI/cilium.md | 248 +++++++++++++++++++++ docs/operations/cgroups.md | 85 +++++++ These seems new, it's not just moved around, right ?

[Payback159]

I also noticed that those two files are recognized as new. Both files were already there.

The only new content is the script. Everything else is just an adaption of existing documentation and scripts to work with the new structure.

Except in cgroups.md I recognized that there are some new lines which are coming form another PR of mine. (But the file also was already there I only added some new lines)

I am going to fix those lines. So that the documentation get's updated with the PR. It doesn't make sense to document the feature before PR is getting merged.

[VannTen]

On closer inspection, it looks like you didn't remove the original files (docs/cilium.md and docs/cgroups.md) files, that would explain why they appear as new.

[Payback159]

Thanks for the hint. I have now deleted the original cilium.md file and moved the original cgroup.md file to the operations folder, now the git diff also fits and the new lines from another PR in the cgroup.md file are also cleaned up.

[VannTen]

End result looks good.

I have three more things:

• could you mark the sidebar as generated so the diff does not show up in PRs ? This is helpful for future docs review. (see https://docs.github.com/en/repositories/working-with-files/managing-files/customizing-how-changed-files-appear-on-github )
• could you also add a pre-commit hook (using the check-readme-versions as an example, for instance) to block PR which did not run a script and ease running it for everyone. This should ensure the sidebar does not go out of sync.
• Once that's done, it would be helpful to reorder your commits, grouping something like this:
  1. Modify the doc structures and the internal links, so there is not an intermediate commit with dead links
  2. add scripts and automation
  3. add result of script (so the generated sidebar).

[Payback159]

Hi @VannTen,

thanks for the feedback. Especially the hint about the .gitattribute, I didn't know that yet... nice feature.

I restructured the commits and added a pre-commit-hook that checks if a file was added in the docs folder, if that is the case it checks if the _sidebar.md was also modified as part of the commit.

If not, there is a recommendation for the developer to execute the gen_docs_sidebar.sh.

[VannTen]

I restructured the commits and added a pre-commit-hook that checks if a file was added in the docs folder, if that is the case it checks if the _sidebar.md was also modified as part of the commit.

AFAIK you don't need to explicitly check, pre-commit return a failure if the running hook modified files, to accommodate formatters and that kind of stuff. So just running the sidebar generation script in the pre-commit should be enough.

[Payback159]

Yes I thought about it but after my research about pre-commit hooks I had the feeling that it is possible but not recommended/best-practice if a pre-commit hook itself changes something, so I went the way of implementing an additional check.

But for me it is also ok if I execute the generation script and pre-commit then aborts. I had seen this behavior because the "end of file check" also modifies data and therefore the pre-commit run failed for me.

Hopefully I'll get around to it tonight, then I'll remove the check-script from the PR and add the sidebar generation script as a pre-commit hook instead.

[VannTen]

On Thu, May 16, 2024 at 03:53:06AM -0700, Alexander wrote: Yes I thought about it but after my research about pre-commit hooks I had the feeling that it is possible but not recommended/best-practice if a pre-commit hook itself changes something, so I went the way of implementing an additional check. But for me it is also ok if I execute the generation script and pre-commit then aborts. I had seen this behavior because the "end of file check" also modifies data and therefore the pre-commit run failed for me.

Yeah I think we already have multiples hooks which operates that way I believe (including the ci-table generation I believe). I don't know if it's a best-practice or not in according to the pre-commit author, but it's a maintenance win for us (one script, two purposes, and it does not go out of sync), so I think it's the best option.

[Payback159]

Yes, I agree. I have removed the check script and added the generation script to the hooks. :-)

[VannTen]

/lgtm
/label tide/merge-method-merge
I restarted the CI (which had problem again 🙄 ), this should merge now
Thanks ! 👍

[VannTen]

Not sure where the merge conflict come from (or if Github is falsely detecting one), but try to rebase ?

[VannTen]

/lgtm

[Payback159]

Thanks @yankay @VannTen