Self-consistency in Documentation

[finanalyst]

(See #2542)
The pod sources that document Perl 6 are in two subdirectories (doc/Language/ and doc/Type/), approximately 360 files. (We can ignore doc/programs/ with three files at present). I call these primary files. Site generation produces over 2300 files in the html/ directory.

This process is highly inefficient and slow. Furthermore, the obscure manner in which these files are generated severely hampers any progress in developing the website or decoupling the documentation from the software that generates the website.

In place of the initial directories doc/Language/,doc/Type/ we end up with html/language/, html/type/ AND html/routine/, html/syntax/

The issue I want to highlight here is that files in html/routine/ and html/syntax/ ONLY exist as derivatives of doc/Language/ and doc/Type/. I call them 'secondary' files. If there is a change in eg doc/Language/str.pod6, it could be reflected by a new file in the html/syntax/ directory, or the disappearance of a file in html/routine/.

Since secondary files are derived from primary files, making any references to secondary files will risk the link being broken, and it is difficult to reconstruct how to fix a link to a secondary file that may no longer exist. I hope everyone agrees that this is not an optimal policy. I repeat there is nothing in a secondary file that does not exist in a primary file.

For whatever reason, there are links in the primary files to secondary files. There are also links outside the perl6.org domain to secondary files.

I have proposed privately to other developers, and do so publicly here:

1. remove all links in primary files to secondary files (eg. any link to a file of the form html/syntax/xxxx.html#yyyy is replaced by its primary source, which is quoted in the secondary file itself)
2. explicitly remove any implied previous guarantee that if a secondary file is generated, it will continue to be generated in the web site.
   HOWEVER, since there was an implied guarantee in the past, I suggest that
3. any and all links to secondary sources in files outside the perl6.org domain, such as blog posts, advent calendar entries, and so on, be collated into .htaccess and nginx config files. These links can then be automatically directed to the relevant primary source. Furthermore, if an author discovers that a link that once worked no longer does so, the target can be added to the .htaccess file reviving the original link.

I am working on (1) and (3) and I will post a PR in due course.

Would it be possible for anyone who vehemently objects to this idea please explain why (2) would be a wrong step?

[JJ]

First, thanks a lot for wording this out, and allow me to make a small change so that auto-links appear.

1. There is nothing in a 2ary file that is not in a 1ary file is, at face value, true. But many secondary files are created by combining content from several primary files.
2. We need to maintain current URLs, including fragments. If we are going to simply eliminate secondary files (which we probably can't, at least some of them, see above), we need to rewrite incoming links so that they point to the new URL.
3. Nobody is vehemently objecting to anything, because available resources, as you can see, are not too plentiful. But 2. is very important for making people find the pages and maintain search engine ranks. If all of a sudden many URLs disappear, they will still be in the search engine for a while and people will simply get lost. This does not imply keeping the URLs (because you can always rewrite), but it does imply to keep the content, even if it's in a different URL that can be mapped. There are at least two type of files that are collated from sources: the above mentioned routine files, and the (admittedly probably useless: Eliminate disambiguation from htmlify.p6 #2499) disambiguation files.
   Still, the problem is, as you mention, that we don't have a functional model of how docs are generated. That is why Create a self-contained mini-docs site #2529 has been proposed, so that the tests become spec, and also we can test any change to the site without running it through the existing 300+ files.

[finanalyst]

I am not 'yet' suggesting that we eliminate these secondary files from the documentation system.

In this issue, I am strongly recommending for the purpose of robust design, that links are made to files that we define to be 'primary' and which are subject to a review process.

The review process is, I think, the crucial difference between 'primary' and 'secondary' files.

When any html file that is generated from a source in the doc/Language, doc/Programs or doc/Type namespace is referenced (viewed in a browser), a pencil button becomes visible in the top right corner. The underlying pod can then be edited. When any file in html/routine or html/syntax spaces is rendered in a browser, it cannot be edited.

When a file is edited, the change is registered in the github system, and the commit can be reversed.

This means that the doc/Language, doc/Type and doc/Program files are subject to review. Others are not.

[JJ]

El mar., 15 ene. 2019 a las 4:20, Richard Hainsworth (< [email protected]>) escribió:

I am not 'yet' suggesting that we eliminate these secondary files from the documentation system. In this issue, I am strongly recommending for the purpose of robust design, that links are made to files that we define to be 'primary' and which are subject to a review process. The review process is, I think, the crucial difference between 'primary' and 'secondary' files. When any html file that is generated from a source in the doc/Language, doc/Programs or doc/Type namespace is referenced (viewed in a browser), a pencil button becomes visible in the top right corner. The underlying pod can then be edited. When any file in html/routine or html/syntax spaces is rendered in a browser, it cannot be edited.

That's not a big problem. As a matter of fact, we have the link to the original source. We could just make the pencil point to that.

When a file is edited, the change is registered in the github system, and the commit can be reversed. This means that the doc/Language, doc/Type and doc/Program files are subject to review. Others are not.

As you say, they are secondary, as in, they have already been reviewed (in their primary form)

[coke]

@finanalyst - you've got this sorted for raku/doc-website now, yes? If there's anything left, please open a followup ticket there.