tools: build doc/api/all.html by combining generated HTML #21568

rubys · 2018-06-27T18:18:01Z

Combine the toc and api contents from the generated doc/api/*.html
files. This ensures that the single page version of the documentation
exactly matches the individual pages.

Fixes #20100

This pull request differs from #21544 in that it is implemented using regular expressions and has no dependencies. See #21544 (comment) for more context.

If this pull request is merged, the other will be deleted.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
documentation is changed or added
commit message follows commit guidelines

Trott · 2018-06-28T19:01:14Z

@TimothyGu @vsemozhetbyt

Trott · 2018-06-29T20:53:43Z

@nodejs/documentation It would be a good idea for someone to build all.html with this and compare it to what's in master. The diff should show some broken links being fixed but otherwise there should be no significant changes.

rubys · 2018-06-30T01:01:29Z

I've made a few fixes to reduce the differences, rebased to master, and here is a current set of differences: https://gist.github.com/rubys/bae70ed087efec625ac04823aaaf803e

Summary:

title is different; instead of picking the first title the new title is the part that is common to all titles. This could be changed, but I would still not recommend that it be the first title.
Some whitespace changes - nothing that would affect rendering
No include comments; something like that could be added if required, but likely would differ in that it would make more sense to reference the generated html files instead of the md source.
An odd change in api stability section; for reasons I don't understand the generation of this section added links previously. Links that are not present on the documentation page. Compare the current node documentation as published publicly: all.html vs documentation.html.
The previously generated all.html would sometimes link to definitions within the all.html page, and other times link to definitions on the separate page, and sometimes link to the wrong thing entirely. This has been changed to always link within the same page, and always to exactly what was linked to in the separate pages.

vsemozhetbyt · 2018-06-30T10:04:14Z

The same results here)

An odd change in api stability section...

This is OK. Due to this code:

node/tools/doc/html.js

Lines 209 to 216 in 1bf42f4

    
           // Do not link to the section we are already in. 
        
           const noLinking = filename === 'documentation' && 
        
             heading !== null && heading.text === 'Stability Index'; 
        
           token.text = `<div class="api_stability api_stability_${number}">` + 
        
             (noLinking ? '' : 
        
               '<a href="documentation.html#documentation_stability_index">') + 
        
             `${prefix} ${number}${noLinking ? '' : '</a>'}${explication}</div>` 
        
             .replace(/\n/g, ' ');

these links were skipped in documentation.html but wrongly reappeared in all.html. The new result is more consistent, I think.

vsemozhetbyt · 2018-06-30T10:11:20Z

tools/doc/allhtml.js

+let contents = '';
+let apicontent = '';
+
+// Identify files that should be skipped.  As files are processed, they


Nit: extra space before "As".

vsemozhetbyt · 2018-06-30T10:31:29Z

tools/doc/allhtml.js

+  .replace('api-section-_toc', 'api-section-all')
+  .replace('data-id="_toc"', 'data-id="all"');
+
+// clean up the title.


clean -> Clean ?

vsemozhetbyt · 2018-06-30T10:32:25Z

tools/doc/allhtml.js

+  '<ul>\n' + contents + '</ul>\n' +
+  all.slice(tocStart.index + tocStart[0].length);
+
+// Replace apicontent with the contenated set of apicontents from each source.


contenated -> concatenated?

Combine the toc and api contents from the generated doc/api/*.html files. This ensures that the single page version of the documentation exactly matches the individual pages. Fixes nodejs#20100

rubys · 2018-06-30T11:38:09Z

Can somebody spot the CI error for me? I don't see it. I searched for 'error' and 'fail'.

vsemozhetbyt · 2018-06-30T11:40:59Z

You can also search for "not ok" in raw log: https://api.travis-ci.com/v3/job/132202841/log.txt
The test failure seems unrelated:
not ok 2146 es-module/test-esm-loader-invalid-format

vsemozhetbyt · 2018-06-30T11:43:04Z

Full CI: https://ci.nodejs.org/job/node-test-pull-request/15681/

vsemozhetbyt · 2018-06-30T11:57:24Z

Refs: #21605

vsemozhetbyt · 2018-06-30T12:23:13Z

#21605 is landed.
New full CI: https://ci.nodejs.org/job/node-test-pull-request/15684/

Trott · 2018-06-30T23:21:27Z

And again... CI: https://ci.nodejs.org/job/node-test-pull-request/15695/

vsemozhetbyt · 2018-07-02T15:13:28Z

I shall land in some hours if there are no objections.

vsemozhetbyt · 2018-07-02T18:36:28Z

Landed in f85962f
Thank you!

Combine the toc and api contents from the generated doc/api/*.html files. This ensures that the single page version of the documentation exactly matches the individual pages. PR-URL: #21568 Fixes: #20100 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Vse Mozhet Byt <[email protected]>

vsemozhetbyt · 2018-07-02T18:44:50Z

Maybe we can do the same for the all.json so that we could completely rid of all.md. This can be done by JSON.parse() + combine new object + JSON.stringify() or just manually concatenate the new stringified serialization, not sure what would be easier or more effective. This may be semver-major as it may change the JSON structure users rely on programmatically.

rubys · 2018-07-02T19:15:39Z

@vsemozhetbyt in the process, we could also get rid of preprocess.js and therefore processIncludes. Please open an issue and I'll work on a pull request.

vsemozhetbyt · 2018-07-02T19:57:27Z

We usually do not require an issue for a PR. Feel free to pace as you feel comfortable)

Trott · 2018-07-02T23:56:50Z

Awesome to see "Contributor" on @rubys's comments etc. everywhere now. 🎉

Combine the toc and api contents from the generated doc/api/*.html files. This ensures that the single page version of the documentation exactly matches the individual pages. PR-URL: #21568 Fixes: #20100 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Vse Mozhet Byt <[email protected]>

nodejs-github-bot added build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations. tools Issues and PRs related to the tools directory. labels Jun 27, 2018

rubys changed the title ~~tools: build doc/api/all.html by combining generated HTML #21544~~ tools: build doc/api/all.html by combining generated HTML Jun 27, 2018

vsemozhetbyt mentioned this pull request Jun 28, 2018

Produce HTML documentation using unified/remark/rehype #21490

Closed

3 tasks

rubys mentioned this pull request Jun 29, 2018

tools: build doc/api/all.html by combining generated HTML #21544

Closed

3 tasks

jasnell approved these changes Jun 29, 2018

View reviewed changes

rubys force-pushed the build_api_all_from_generated_html2 branch from 338d466 to 0b9ff5f Compare June 30, 2018 00:36

vsemozhetbyt approved these changes Jun 30, 2018

View reviewed changes

tools: build doc/api/all.html by combining generated HTML nodejs#21544

6dac7db

Combine the toc and api contents from the generated doc/api/*.html files. This ensures that the single page version of the documentation exactly matches the individual pages. Fixes nodejs#20100

rubys force-pushed the build_api_all_from_generated_html2 branch from 0b9ff5f to 6dac7db Compare June 30, 2018 11:14

vsemozhetbyt closed this Jul 2, 2018

targos mentioned this pull request Jul 3, 2018

v10.6.0 proposal #21629

Merged

rubys mentioned this pull request Jul 3, 2018

tools: build all.json by combining generated JSON #21637

Closed

3 tasks

vsemozhetbyt mentioned this pull request Jul 9, 2018

Creating a branch of the Docs using RunKit #21723

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tools: build doc/api/all.html by combining generated HTML #21568

tools: build doc/api/all.html by combining generated HTML #21568

rubys commented Jun 27, 2018

Trott commented Jun 28, 2018

Trott commented Jun 29, 2018

rubys commented Jun 30, 2018 •

edited

Loading

vsemozhetbyt commented Jun 30, 2018

vsemozhetbyt Jun 30, 2018

vsemozhetbyt Jun 30, 2018

vsemozhetbyt Jun 30, 2018

rubys commented Jun 30, 2018

vsemozhetbyt commented Jun 30, 2018

vsemozhetbyt commented Jun 30, 2018

vsemozhetbyt commented Jun 30, 2018

vsemozhetbyt commented Jun 30, 2018

Trott commented Jun 30, 2018

vsemozhetbyt commented Jul 2, 2018

vsemozhetbyt commented Jul 2, 2018

vsemozhetbyt commented Jul 2, 2018

rubys commented Jul 2, 2018

vsemozhetbyt commented Jul 2, 2018

Trott commented Jul 2, 2018 •

edited

Loading

tools: build doc/api/all.html by combining generated HTML #21568

tools: build doc/api/all.html by combining generated HTML #21568

Conversation

rubys commented Jun 27, 2018

Checklist

Trott commented Jun 28, 2018

Trott commented Jun 29, 2018

rubys commented Jun 30, 2018 • edited Loading

vsemozhetbyt commented Jun 30, 2018

vsemozhetbyt Jun 30, 2018

Choose a reason for hiding this comment

vsemozhetbyt Jun 30, 2018

Choose a reason for hiding this comment

vsemozhetbyt Jun 30, 2018

Choose a reason for hiding this comment

rubys commented Jun 30, 2018

vsemozhetbyt commented Jun 30, 2018

vsemozhetbyt commented Jun 30, 2018

vsemozhetbyt commented Jun 30, 2018

vsemozhetbyt commented Jun 30, 2018

Trott commented Jun 30, 2018

vsemozhetbyt commented Jul 2, 2018

vsemozhetbyt commented Jul 2, 2018

vsemozhetbyt commented Jul 2, 2018

rubys commented Jul 2, 2018

vsemozhetbyt commented Jul 2, 2018

Trott commented Jul 2, 2018 • edited Loading

rubys commented Jun 30, 2018 •

edited

Loading

Trott commented Jul 2, 2018 •

edited

Loading