docs: reorganize sphinx docs for better discoverability

[brian-smith-tcril]

continued from #766 (rebasing autoclosed that one)

This reorganizes the docs quite a bit, the full before/after TOC trees are available here: https://gist.github.com/brian-smith-tcril/4fc2edff60b171efc434f2748003e191

I also have this deployed to GH pages from my fork, to see it live go here: https://brian-smith-tcril.github.io/tutor/

Looking forward to feedback!

[brian-smith-tcril]

@regisb this one is ready for review!

[regisb]

Hey @brian-smith-tcril, thanks again for this PR! I really like the reog. What I like less it that it highlights all the places where the docs are badly written or straight up wrong 😅 but that's not on you of course. In this PR we focus just on the layout and chaptering, and not so much on the actual contents. We'll definitely have to do an exhaustive review after we merge this.

I'm in agreement with most changes, with the exception of the removal of the tutorials and the "what next" section. If you have a strong opinion on those changes, maybe we should have a conversation? Let me know in the comments.

[regisb]

Tutorials are actually pretty important, and we must keep them in a separate chapter. The reason for that is that tutorials really have a different format than the rest of the docs. Tutorials are walkthrough instructions, separate from explanations or reference pages.

This is a big change, so I assume there was some strong motivation? If you disagree maybe we should have a conversation about this.

[brian-smith-tcril]

I understand they have a different format, but there is a lot of important documentation that only exists in a tutorial.

If we take "Creating a Tutor Plugin" as an example, in the current docs it is nested under

• User Guide
  • Tutorials
    • Open edX customization
      • Creating a Tutor plugin

In the reorg I moved it to

• Developer Guide
  • Tutor plugin development
    • Creating a Tutor plugin

My thought process here was if I were coming to the docs with the goal of creating a tutor plugin, I would have no idea that I should look that deep into the Tutorials section. In fact, that exact scenario is what inspired me to start working on this reorg in the first place 😂

I understand the desire to separate them based on the format of the information, but it seems that is currently happening at the expense of discoverability.

I'm very open to ideas though! Any thoughts on how to keep the format distinction without hiding the information away?

[regisb]

I understand. After the proposed reorg, the "User Guide -> Plugins" becomes easy to find and the tutorial is mentioned in the first section. We can certainly help by adding more information to that page. In particular, we should mention how a plugin is detected (either from ~/.local/share/tutor-plugins or the "tutor.plugin.v1" entrypoint). This could be done in a dedicated "Creating a Tutor plugin" section in that page; the content would be different from the tutorial with the same name, though.

Moreover, I propose that we surface the tutorial by moving them from under "user guide" to their own dedicated top-level section. Thus we would have "Tutorials -> Open edX customization -> Creating a Tutor plugin."

[brian-smith-tcril]

I propose that we surface the tutorial by moving them from under "user guide" to their own dedicated top-level section.

Overall this sounds like a reasonable path forward, but I'm not so sure about the specifics here.

After the proposed reorg, the "User Guide -> Plugins" becomes easy to find and the tutorial is mentioned in the first section.

This doesn't really help in my opinion. I don't think I'm alone in my strategy of only looking at the navigation bar until I find something that seems related to what I'm trying to do. In that case, I would have clicked on plugins and seen

or

decided that information on creating a tutor plugin didn't exist there, and continued clicking around the navbar.

Thus we would have "Tutorials -> Open edX customization -> Creating a Tutor plugin."

This also doesn't quite solve the issue in my opinion. Coming into the docs thinking "I want to make a tutor plugin" how would I know to look under "Open edX customization"?

As a dev trying to make a new plugin, I was thinking, "I want to make a plugin. Show me an example and tell me how it works." The example plugins are very simple and don't cover close to as much of the functionality as the tutorial does, and also don't address the "tell me how it works" part. I more or less wanted a beeline to https://docs.tutor.overhang.io/tutorials/plugin.html#final-result, and then to scroll around the page to get some context as to what each piece of the final result is doing.

With my proposed reorg that beeline exists: "Developer Guide -> Tutor plugin development -> Creating a tutor plugin" If we move to having top level "Tutorials" I think we could make that beeline with

• Tutorials
  • Plugins
    • Creating a Tutor plugin

or even just

• Tutorials
  • Creating a Tutor plugin

Another example is https://docs.tutor.overhang.io/tutorials/nightly.html. I would have no idea to look under "Tutorials -> Open edX customization" to find information on running nightly.

Even with tutorials as a top level thing it seems likely information would be hiding in there (unless we removed the existing subcategories of "Open edX customization" and "System administration" entirely so every tutorial was visible without clicking)

[regisb]

If we move to having top level "Tutorials" I think we could make that beeline with

Tutorials
    Plugins
        Creating a Tutor plugin

I agree, let's do that.

I also agree that in many cases the information is indeed hard to find -- the nightly docs is a good example that you pointed out. But this is as much a problem of reorganising the section as putting more words in the docs. Can I suggest that you list the place in the docs that, in your opinion, lack some information? You can either create a GitHub issue or add a big TODO in the file where you would expect to find more words. Depending on the amount of work, we might resolve these issues in this PR or later.

[brian-smith-tcril]

The more I think about this the less I like it. By splitting up information at the top-level based on how it's presented, navigation becomes much more convoluted.

I understand the principles of Diátaxis suggest organizing docs into Tutorials/How-Tos/Explanation/Reference, but I think that separation makes sense within other categories.

If we take the 4 types and turn them into user statements, we arrive at

Type	Statement
Tutorials	I want to learn about _____.
How-To Guides	How do I do something related to _____?
Explanation	I want to better understand _____.
Reference	I need information about _____.

If we fill in the blank with "Tutor," then it makes sense to have a top level split based on Diátaxis type. I don't, however, think most people coming to the docs would fill in the blank that way.

I know I came to the docs filling in that blank with "Tutor plugins." I know plenty of people come to the docs filling in that blank with "running Open edX."

With this in mind, I think it makes sense to make sure the high level categories line up with how people would "fill in the blank," and then find a way to clearly differentiate the information by Diátaxis type within said categories.

Thinking about this in practical terms, I've asked myself, "what should be visible in the navigation sidebar when viewing this?" I believe people reading a tutorial about plugins are more likely to want to see more information about plugins in the sidebar than other tutorials.

I'm very open to other thoughts and feelings on this, I just wanted to make sure I expressed mine clearly before making changes I was unhappy with.

[regisb]

Hey, sorry about the lack of reply. I was busy with the Palm upgrade for the past couple of days.

I think you really shouldn't be making changes that you're unhappy with. You want to make it as easy as possible for users to discover the plugins tutorial, and that's perfectly reasonable. You argue that many users navigate their way through the docs mostly by using the navigation sidebar, and that the first point of entry for users interested in plugins is the "Plugins" section -- again, perfectly reasonable.

What I want to avoid is to distribute the tutorials across different sections. There are a two main reasons for that:

1. Some tutorials do not belong to one specific section.
2. The Tutorials are in some way a kitchen sink where we can dump information about many different topics (similar to the Django "How-To" guides).

To reconcile your position and mine, can we add an entry to the plugins TOC that would send users directly to the plugins tutorial?

[MaferMazu]

Hello there :D I was checking the doc for an issue I have assigned, and finding the info about creating a tutor plugin was difficult. I think the @regisb idea of adding an entry in Plugin that redirects to plugin tutorials is a good idea.

Btw @brian-smith-tcril, thanks for this PR is fantastic.

[brian-smith-tcril]

To reconcile your position and mine, can we add an entry to the plugins TOC that would send users directly to the plugins tutorial?

What would the navigation sidebar look like in this scenario? Would we have both plugins and tutorials expanded when someone is reading the plugins tutorial?