Docs/re structure config docs

[merelcht]

Description

Development notes

• Added an introduction to configuration in Kedro: what do we mean by project configuration
• Divided the page into "basic configuration" and "advanced configuration"
• Each page has short descriptions about core concepts, the rest of the content I've rephrased as "how to ..." questions with answers
• Put credentials and parameters on two separate pages.

Remaining tasks/decisions to make

• Should "How to read configuration from a compressed file?" stay in the basic configuration page or go to advanced config? JCS: Yes, I think it should stay where it is
• Add intro to "advanced configuration" page JCS: Done
• The TemplatedConfigLoader still has quite a lot of content with the "Provide template values through globals" section, but I feel like if we move that to the "how to.." section it's not clear at all what this loader is about or why you'd use it. Any suggestions on how to make the intro to the TemplatedConfigLoader shorter but still clear how to do the templating? JCS: I feel like this section is not really explaining "why" to the reader. So it starts as follows:

"When using the TemplatedConfigLoader you can provide values in the configuration template through a globals file or dictionary."

But what is the use case by which I'd need this? (I can't answer that, sorry). Would it be possible to turn this into more of a "how to" by adding a sentence or two that explains the motivation that a user would have in using a globals file? So retitling from "Provide template values through globals" to something like "How to provide a dynamic set of settings using global values" (totally made up). Then give a sentence or two as suggested and launching into the details? I still think it's fine where it is, but you can make it more useful by positioning it better.

• Should the chapters about credentials and parameters be under the basic or advanced configuration or on separate pages as I've done now? : JCS: I like the separate pages

Checklist

• Read the contributing guidelines
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added a description of this change in the RELEASE.md file
• Added tests to cover my changes

[stichbury]

• I think I'd be tempted to split the page and have the Configuration basics on one, and the "How to" information about parameters, Credentials and the loader on another page.

• Also, to that effect, I'd make the sections in that section page much more focussed on "How to..." so you'd have "How to load credentials", "How to use AWS credentials", "How to specify a loader" kind of thing.

• I'd like a set of FAQs on the basics page too -- "Why use this?" "Can I load X?" with answers, and then an additional set of questions which you point through to the relevant information on the other page (point to the anchors of the relevant subsections)

• Also, I think putting some of the code snippets behind expandables would help

[astrojuanlu]

Discussed synchronously between @merelcht, @stichbury and I:

• Let's try to avoid mixing explanations with how-to guides in the Diátaxis spirit https://diataxis.fr/
• Here "config" refers to "whatever is under conf/", therefore not settings.py and not pyproject.toml (they can be treated separately)
• From an Information Architecture perspective this can be tricky because "config" includes the catalog, but we have a separate set of pages for the catalog because it's central to using Kedro
• On the other hand, for the content here, the baseline is that "regular users don't need any of this", so let's make it clear at the top and maybe let's rename this whole page to "Advanced Configuration" (or try to separate the really typical use cases from the advanced ones)
• The examples are good, but we should reframe them in terms of use cases, e.g. "how do I pass Python objects to a catalog entry" rather than "how do I instantiate a new TemplatedConfigLoader"

[merelcht]

@stichbury @astrojuanlu I've made a first pass of dividing the configuration docs into basic and advanced chapters. I think the basics are in pretty good shape now. Can you have a look and tell me what you think? (ignore the links I haven't fixed them yet in case the pages still change)

You can ignore the advanced page for now. I also need to decide where the sections on credentials and parameters should go. I put them in separate pages now, but let me know if you have ideas for that.

[astrojuanlu]

I really like the direction this is taking! 💯

[stichbury]

I've made a number of commits to answer some of the queries/suggestions you'd left above (and some notes above to summarise). I think there's probably a bit left for you, Merel, before we hand over to @juan Luis for a final review (or you can pass it back to me to make another round of changes). It's looking SO. MUCH. BETTER. Thank you 🙏

• One thing I think we could do is provide links in the basic docs to anything in the more advanced docs that a reader may want to find out more about. For example, we state the configuration loading rules in the basic docs and then have a How To in the advanced that covers "How to bypass the configuration loading rules". So it would be handy to point across. I didn't do this because it would be a bit of a guess on my part as to which How To sections apply where, but I'm hoping that's easy for you.
• Another thing outstanding is to fix broken internal linking and to add links to the anchors of the how-to sections (I've created mini table of contents on each page). Not done this yet because we may yet revise the subhead wording, but I'm happy to do that as a final step.
• Are there any diagrams we could "commission" or do in Mermaid to summarise loading rules? I can't help feeling a graphic would be useful.

[astrojuanlu]

Read the whole thing, fantastic job @merelcht 👏🏽 I did learn a lot! Left a few style comments but other than that this is a great addition

[ankatiyar]

+1 on @stichbury's comment about providing links in the basic docs to related sections in the advanced config page that might be of interest.
Read through the docs, it looks great! 💯

[merelcht]

I fixed all the links and linked the individual config pages to each other where it made sense. I guess over to you one more time @stichbury / @astrojuanlu for a final review.

[stichbury]

I'm really happy with how this turned out -- thanks @merelcht! 🏆