Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v10.x backport] doc,tools: get altDocs versions from CHANGELOG.md #32642

Closed
wants to merge 6 commits into from
Closed

[v10.x backport] doc,tools: get altDocs versions from CHANGELOG.md #32642

wants to merge 6 commits into from

Conversation

richardlau
Copy link
Member

@richardlau richardlau commented Apr 3, 2020 •
edited
Loading

Backports of the following PRs to v10.x-staging:

  1. doc,tools: get altDocs versions from CHANGELOG.md #27661
  2. test: fix flaky doctool and test #29979
  3. tools: make doctool work if no internet available #30214
  4. tools: add NODE_TEST_NO_INTERNET to the doc builder #31849
  5. tools: only fetch previous versions when necessary #32518
  6. test: fix tool path in test-doctool-versions.js #32645

The first and fifth of these required manual backporting while the middle three in the sequence cherry-picked cleanly. Does Backport-PR-URL: metadata go on all of the commits in this backport PR or only the ones that didn't cherry-pick cleanly (@nodejs/release)?

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations. tools Issues and PRs related to the tools directory. v10.x labels Apr 3, 2020
@richardlau richardlau mentioned this pull request Apr 3, 2020
Closed
3 tasks
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/30421/

@richardlau
Copy link
Member Author

🤔 Test is failing across the CI. Passes locally for me, and when I log into the CI host and run manually:

[iojs@test-ibm-rhel7-s390x-4 rhel7-s390x]$ ./node --version
v10.19.1-pre
[iojs@test-ibm-rhel7-s390x-4 rhel7-s390x]$ NODE_DEBUG=test ./node test/doctool/test-doctool-versions.js
TEST 62870: /home/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel7-s390x/test/.tmp/versions.json
TEST 62870:
TEST 62870:
TEST 62870: [ { num: '13.x' },
  { num: '12.x', lts: true },
  { num: '11.x' },
  { num: '10.x', lts: true },
  { num: '9.x' },
  { num: '8.x' },
  { num: '7.x' },
  { num: '6.x' },
  { num: '5.x' },
  { num: '4.x' },
  { num: '0.12.x' },
  { num: '0.10.x' } ]
[iojs@test-ibm-rhel7-s390x-4 rhel7-s390x]$

@richardlau
Copy link
Member Author

richardlau commented Apr 3, 2020 •
edited
Loading

Okay it's a bug in the original PR if NODE_TEST_DIR is set to a path outside of the test directory but wasn't spotted because doctool tests are excluded from default in master but not in v10.x-staging. I'll PR a fix to master and backport it into this PR for 10.x.

@richardlau richardlau added the blocked PRs that are blocked by other issues or PRs. label Apr 3, 2020
@richardlau
Copy link
Member Author

Blocked on #32645 landing on master.

richardlau and others added 6 commits April 6, 2020 05:43
@richardlau
doc,tools: get altDocs versions from CHANGELOG.md
b0a4d45 
Parse `CHANGELOG.md` for versions of Node.js used by the documentation
feature `View another version` so that we don't have to manually update
the list when we cut a new version or transition a release to LTS.

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#27661
Reviewed-By: Rich Trott <[email protected]>
@Trott @richardlau
test: fix flaky doctool and test
ed69a5f 
Doctool tests have been failing a lot in CI on Win2008 R2. It appears
async functions and callback-based functions are being used in
combination such that the callback-based function cannot guarantee that
it will invoke its callback. Convert the callback-based functions to
async functions so we have one paradigm and reliable results.

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#29979
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Jeremiah Senkpiel <[email protected]>
@richardlau
tools: make doctool work if no internet available
ac9127e 
Allow doctool to fallback to use local files if not building a release
build.

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#30214
Fixes: nodejs#29918
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Jiawen Geng <[email protected]>
@joyeecheung @richardlau
tools: add NODE_TEST_NO_INTERNET to the doc builder
2c3c6a0 
At the moment the doc builder tries to access the internet
for CHANGELOG information and only falls back to local sources
after the connection fails or a 5 second timeout. This means
that the doc building could take at least 7 minutes on a
machine with hijacked connection to Github for useless network
attempts. This patch adds a NODE_TEST_NO_INTERNET environment
variable to directly bypass these attempts so that docs can be
built in reasonable time on a machine like that.

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#31849
Fixes: nodejs#29918
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
@richardlau
tools: only fetch previous versions when necessary
df5d115 
Refactor the logic for working out the previous versions of Node.js for
the API documentation so that the parsing (including the potential https
get) happens at most once per build (as opposed to the current once per
generated API doc).

Signed-off-by: Richard Lau <[email protected]>

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#32518
Fixes: nodejs#32512
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Myles Borins <[email protected]>
@richardlau
test: fix tool path in test-doctool-versions.js
91ea203 
Path to the versions tool tested by test-doctool-versions.js would
be incorrect if the test temporary directory was redirected (e.g.
via NODE_TEST_DIR) outside of `test/`.

Signed-off-by: Richard Lau <[email protected]>

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#32645
Refs: nodejs#32518
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Shelley Vohr <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
@richardlau richardlau removed the blocked PRs that are blocked by other issues or PRs. label Apr 6, 2020
@richardlau richardlau force-pushed the backport-32518-to-10 branch from 3d5432a to 91ea203 Compare April 6, 2020 09:45
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/30493/

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Apr 6, 2020 •
edited by BethGriggs
Loading

CI: https://ci.nodejs.org/job/node-test-pull-request/30495/ ✅

BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
@richardlau @BethGriggs
doc,tools: get altDocs versions from CHANGELOG.md
0177464 
Parse `CHANGELOG.md` for versions of Node.js used by the documentation
feature `View another version` so that we don't have to manually update
the list when we cut a new version or transition a release to LTS.

Backport-PR-URL: #32642
PR-URL: #27661
Reviewed-By: Rich Trott <[email protected]>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
@Trott @BethGriggs
test: fix flaky doctool and test
1ea70d6 
Doctool tests have been failing a lot in CI on Win2008 R2. It appears
async functions and callback-based functions are being used in
combination such that the callback-based function cannot guarantee that
it will invoke its callback. Convert the callback-based functions to
async functions so we have one paradigm and reliable results.

Backport-PR-URL: #32642
PR-URL: #29979
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Jeremiah Senkpiel <[email protected]>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
@richardlau @BethGriggs
tools: make doctool work if no internet available
ac1ea73 
Allow doctool to fallback to use local files if not building a release
build.

Backport-PR-URL: #32642
PR-URL: #30214
Fixes: #29918
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Jiawen Geng <[email protected]>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
@joyeecheung @BethGriggs
tools: add NODE_TEST_NO_INTERNET to the doc builder
3756be8 
At the moment the doc builder tries to access the internet
for CHANGELOG information and only falls back to local sources
after the connection fails or a 5 second timeout. This means
that the doc building could take at least 7 minutes on a
machine with hijacked connection to Github for useless network
attempts. This patch adds a NODE_TEST_NO_INTERNET environment
variable to directly bypass these attempts so that docs can be
built in reasonable time on a machine like that.

Backport-PR-URL: #32642
PR-URL: #31849
Fixes: #29918
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
@richardlau @BethGriggs
tools: only fetch previous versions when necessary
a175b8d 
Refactor the logic for working out the previous versions of Node.js for
the API documentation so that the parsing (including the potential https
get) happens at most once per build (as opposed to the current once per
generated API doc).

Signed-off-by: Richard Lau <[email protected]>

Backport-PR-URL: #32642
PR-URL: #32518
Fixes: #32512
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Myles Borins <[email protected]>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
@richardlau @BethGriggs
test: fix tool path in test-doctool-versions.js
017909b 
Path to the versions tool tested by test-doctool-versions.js would
be incorrect if the test temporary directory was redirected (e.g.
via NODE_TEST_DIR) outside of `test/`.

Signed-off-by: Richard Lau <[email protected]>

Backport-PR-URL: #32642
PR-URL: #32645
Refs: #32518
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Shelley Vohr <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
@BethGriggs
Copy link
Member

Landed in 2e3d511...017909b

@BethGriggs BethGriggs closed this Apr 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BethGriggs BethGriggs BethGriggs approved these changes

Assignees
No one assigned
Labels
build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations. tools Issues and PRs related to the tools directory.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@richardlau @nodejs-github-bot @BethGriggs @Trott @joyeecheung