crypto: increase maxmem range from 32 to 53 bits #28799

tniessen · 2019-07-21T18:28:41Z

This implements @bnoordhuis' suggestion in order to support larger parameters. The new limit is roughly 8 petabytes, which should be sufficient for the next couple of releases.

Fixes: #28755

cc @nodejs/crypto @michaelsbradleyjr

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

bnoordhuis

LGTM and I appreciate your sense of understatement. :-)

lib/internal/crypto/scrypt.js

doc/api/crypto.md

lib/internal/crypto/scrypt.js

Fixes: nodejs#28755

nodejs-github-bot · 2019-07-22T21:40:17Z

CI: https://ci.nodejs.org/job/node-test-pull-request/24621/

tniessen · 2019-07-23T11:54:39Z

This is now effectively blocked by #28810.

Trott · 2019-07-23T18:45:17Z

This is now effectively blocked by #28810.

Maybe optimistically make the changes here to use the API introduced there? Or, you know, don't if you're not as impatient as I am. :-D

tniessen · 2019-07-23T19:33:08Z

@Trott I already did that, but the tests won't pass right now because the min argument is not on master yet :-) It should be ready to land as soon as #28810 lands.

Trott · 2019-07-23T20:29:39Z

@Trott I already did that

D'oh! Never mind! :-D

Trott · 2019-07-23T21:58:26Z

#28810 landed. Unblocking.

nodejs-github-bot · 2019-07-23T22:45:25Z

CI: https://ci.nodejs.org/job/node-test-pull-request/24647/

Trott · 2019-07-24T04:18:20Z

Landed in 1dc458c

Fixes: nodejs#28755 PR-URL: nodejs#28799 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]>

Notable changes: * assert: * Legacy mode deprecation (`DEP0089`) is revoked (Colin Ihrig) #28892 * crypto: * The `outputLength` option is added to `crypto.createHash` (Tobias Nießen) #28805 * The `maxmem` range is increased from 32 to 53 bits (Tobias Nießen) #28799 * n-api: * Added APIs for per-instance state management (Gabriel Schulhof) #28682 * report: * Network interfaces get included in the report (cjihrig) #28911 * src: * `v8.getHeapCodeStatistics()` is now exported (Yuriy Vasiyarov) #27978 PR-URL: #29017

Notable changes: * assert: * Legacy mode deprecation (`DEP0089`) is revoked (Colin Ihrig) nodejs#28892 * crypto: * The `outputLength` option is added to `crypto.createHash` (Tobias Nießen) nodejs#28805 * The `maxmem` range is increased from 32 to 53 bits (Tobias Nießen) nodejs#28799 * n-api: * Added APIs for per-instance state management (Gabriel Schulhof) nodejs#28682 * report: * Network interfaces get included in the report (cjihrig) nodejs#28911 * src: * `v8.getHeapCodeStatistics()` is now exported (Yuriy Vasiyarov) nodejs#27978 PR-URL: nodejs#29017

* report: modify getReport() to return an Object It's likely that anyone using `process.report.getReport()` will be processing the return value thereafter (e.g., filtering fields or redacting secrets). This change eliminates boilerplate by calling `JSON.parse()` on the return value. Also modified the `validateContent()` and `validate()` test helpers in `test/common/report.js` to be somewhat more obvious and helpful. Of note, a report failing validation will now be easier (though still not _easy_) to read when prepended to the stack trace. - Refs: https://github.com/nodejs/diagnostics/issues/315 PR-URL: https://github.com/nodejs/node/pull/28630 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: replace already elevated Object, Local v8 namespace PR-URL: https://github.com/nodejs/node/pull/28611 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * src, tools: replace raw ptr with smart ptr NodeMainInstance::Create will now returrn an instance of NodeMainInstance in a unique_ptr. PR-URL: https://github.com/nodejs/node/pull/28577 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: add example on how to create __filename, __dirname for esm PR-URL: https://github.com/nodejs/node/pull/28282 Fixes: https://github.com/nodejs/node/issues/28114 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: simplify --debug flags Any use of --debug, --debug=, --debug-brk, or --debug-brk= now triggers an error. That means we can eliminate their aliases with --inspect counterparts and simplify the code. PR-URL: https://github.com/nodejs/node/pull/28615 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> * doc: drop 'for more details' in deprecations The deprecations documentation links to the GitHub issue tracker in several places. This commit makes the text around those links consistent. PR-URL: https://github.com/nodejs/node/pull/28617 Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * readline: use named constant for surrogate checks This commit defines a named constant instead of using a mix of 2 ** 16 and 0x10000 throughout the code. PR-URL: https://github.com/nodejs/node/pull/28638 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> * readline: remove IIFE in SIGCONT handler This commit removes an IIFE in the readline SIGCONT handler that was previously being used to bind `this`. PR-URL: https://github.com/nodejs/node/pull/28639 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> * readline: simplify isFullWidthCodePoint() The non-ICU-based isFullWidthCodePoint() can be simplified to a single `return` statement. This commit removes the extra branching logic. PR-URL: https://github.com/nodejs/node/pull/28640 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> * doc: remove superfluous MDN link in assert.md PR-URL: https://github.com/nodejs/node/pull/28246 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * readline: expose stream API in clearScreenDown() This commit adds an optional callback to clearScreenDown(), which is passed to the stream's write() method. It also exposes the return value of write(). PR-URL: https://github.com/nodejs/node/pull/28641 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> * path: using .relative() should not return a trailing slash Resolving a path against root with `path.relative()` should not include a trailing slash. Fixes: https://github.com/nodejs/node/issues/28549 PR-URL: https://github.com/nodejs/node/pull/28556 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Weijia Wang <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> * path: move branch to the correct location This code branch only makes sense when i === length. Otherwise it'll already be handled. PR-URL: https://github.com/nodejs/node/pull/28556 Fixes: https://github.com/nodejs/node/issues/28549 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Weijia Wang <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> * doc: mark process.report as experimental Everything under process.report is experimental. This commit adds the missing stability index entries. PR-URL: https://github.com/nodejs/node/pull/28653 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> * doc: mark N-API thread-safe function stable The various TSFN APIs are marked as stable, but the TSFN heading itself is still marked as experimental. PR-URL: https://github.com/nodejs/node/pull/28643 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * zlib: do not coalesce multiple `.flush()` calls This is an approach to address the issue linked below. Previously, when `.write()` and `.flush()` calls to a zlib stream were interleaved synchronously (i.e. without waiting for these operations to finish), multiple flush calls would have been coalesced into a single flushing operation. This patch changes behaviour so that each `.flush()` all corresponds to one flushing operation on the underlying zlib resource, and the order of operations is as if the `.flush()` call were a `.write()` call. One test had to be removed because it specifically tested the previous behaviour. As a drive-by fix, this also makes sure that all flush callbacks are called. Previously, that was not the case. Fixes: https://github.com/nodejs/node/issues/28478 PR-URL: https://github.com/nodejs/node/pull/28520 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * src: add cleanup hook for ContextifyContext Otherwise there’s a memory leak left by the context when the Isolate tears down without having run the weak callback. PR-URL: https://github.com/nodejs/node/pull/28631 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> * deps: update acorn to 6.2.0 Includes support for bigint syntax so we can remove the acorn-bigint plugin. PR-URL: https://github.com/nodejs/node/pull/28649 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> * http2: report memory allocated by nghttp2 to V8 This helps the JS engine have a better understanding of the memory situation in HTTP/2-heavy applications, and avoids situations that behave like memory leaks due to previous underestimation of memory usage which is tied to JS objects. Refs: https://github.com/nodejs/node/issues/28088#issuecomment-509965105 PR-URL: https://github.com/nodejs/node/pull/28645 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> * tools: add coverage to ignored files This adds the coverage directory to the .gitignore file. PR-URL: https://github.com/nodejs/node/pull/28626 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Masashi Hirano <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * build,v8: support IBM i Some libraries do not exist on IBM i (OS400). Commit 417c18e introduces these missing libraries. Need to differentiate `AIX` and `OS400`(IBM i). PR-URL: https://github.com/nodejs/node/pull/28607 Reviewed-By: Beth Griggs <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Michael Dawson <[email protected]> * test: fix pty test hangs on aix Some pty tests persistently hung on the AIX CI buildbots. Fix that by adding a helper script that properly sets up the pty before spawning the script under test. On investigation I discovered that the test runner hung when it tried to close the slave pty's file descriptor, probably due to a bug in AIX's pty implementation. I could reproduce it with a short C program. The test runner also leaked file descriptors to the child process. I couldn't convince python's `subprocess.Popen()` to do what I wanted it to do so I opted to move the logic to a helper script that can do fork/setsid/etc. without having to worry about stomping on state in tools/test.py. In the process I also uncovered some bugs in the pty module of the python distro that ships with macOS 10.14, leading me to reimplement a sizable chunk of the functionality of that module. And last but not least, of course there are differences between ptys on different platforms and the helper script has to paper over that. Of course. Really, this commit took me longer to put together than I care to admit. Caveat emptor: this commit takes the hacky ^D feeding to the slave out of tools/test.py and puts it in the *.in input files. You can also feed other control characters to tests, like ^C or ^Z, simply by inserting them into the corresponding input file. I think that's nice. Fixes: https://github.com/nodejs/build/issues/1820 Fixes: https://github.com/nodejs/node/issues/28489 PR-URL: https://github.com/nodejs/node/pull/28600 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> * stream: simplify `.pipe()` and `.unpipe()` in Readable Now we are using `pipes` and `pipesCount` in Readable state and the `pipes` value can be a stream or an array of streams. This change reducing them into one `pipes` value, which is an array of streams. PR-URL: https://github.com/nodejs/node/pull/28583 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> * src: lint #defines in src/node.h A few #defines in src/node.h had inconsistent spacing and tabbing. This commit changes the spacing to be the same style as the rest of the project. PR-URL: https://github.com/nodejs/node/pull/28547 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> * build: remove broken intel vtune support Support for VTune profiling was added in commit a881b53 from November 2015 but has since bitrotted. Remove it. Fixes: https://github.com/nodejs/node/issues/28310 Refs: https://github.com/nodejs/node/pull/3785 PR-URL: https://github.com/nodejs/node/pull/28522 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * src: add missing option parser template for the DebugOptionsParser This allows embedders to run `node::options_parser::Parse` for a `node::DebugOptions`. PR-URL: https://github.com/nodejs/node/pull/28543 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Eugene Ostroukhov <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * test: increase limit for network space overhead test This test imposes a limit on the average bytes of space per chunk for network traffic. However this number depends on VM implementation details, and upcoming changes to V8's array buffer management require a small bump to this limit in this test. PR-URL: https://github.com/nodejs/node/pull/28492 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * http: fix test where aborted should not be emitted PR-URL: https://github.com/nodejs/node/pull/20077 Fixes: https://github.com/nodejs/node/issues/20107 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Matteo Collina <[email protected]> * stream: use readableEncoding public api for child_process PR-URL: https://github.com/nodejs/node/pull/28548 Refs: https://github.com/nodejs/node/issues/445 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: add documentation for createDiffieHellmanGroup PR-URL: https://github.com/nodejs/node/pull/28585 Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: cherry-pick 13a04aba from V8 upstream Original commit message: fix: move V8_EXPORT_PRIVATE marks to prevent unresolvable references This change fixes missing symbol errors in the Windows 10 on ARM build of Node.js. When a whole class is marked for export, all of its members are marked as well. This can be a problem when inline members call undefined yet inline members of other classes: the exported function will contain a reference to the undefined inline function that should be satisfied at link time, but because the other function is inline no symbol will be produced that will satisfy that reference. Clang gets around this by masking inlined class members from export using /Fc:dllexportInlines-. This is why b0a2a567 worked. Node.js' Windows builds use MSVC and so do not have access to this flag. This results in unresolved symbols at link time. Bug: v8:9465 Change-Id: Ief9c7ab6ba35d22f995939eb62a64d6f1992ed85 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1696771 Reviewed-by: Sigurd Schneider <[email protected]> Reviewed-by: Jakob Gruber <[email protected]> Commit-Queue: Sigurd Schneider <[email protected]> Cr-Commit-Position: refs/heads/master@{#62660} Refs: https://github.com/v8/v8/commit/13a04abacd6a15b0b06c9ad08e237af703a57dec PR-URL: https://github.com/nodejs/node/pull/28602 Reviewed-By: João Reis <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Richard Lau <[email protected]> * deps: cherry-pick 721dc7d from node-gyp upstream This change cherry-picks a small set of node-gyp v5.0.0 changes needed to enable Node.js ARM64 Windows builds. Original commit message: Add ARM64 to MSBuild /Platform logic PR-URL: https://github.com/nodejs/node-gyp/pull/1655 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: João Reis <[email protected]> Refs: https://github.com/nodejs/node-gyp/commit/721dc7d3148ab71ee283d9cb15df84d9b87b7efc PR-URL: https://github.com/nodejs/node/pull/28604 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: João Reis <[email protected]> * deps: cherry-pick 91744bf from node-gyp upstream This change cherry-picks a small set of node-gyp v5.0.0 changes needed to enable Node.js ARM64 Windows builds. Original commit message: gyp: add support for Windows on Arm Cherry-pick of https://github.com/refack/GYP/pull/33, supersedes https://github.com/nodejs/node-gyp/pull/1678 until GYP3 is merged. `npm test` passes Change-Id: I2b1e1e03e378b4812d34afa527087793864d1576 PR-URL: https://github.com/nodejs/node-gyp/pull/1739 Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: João Reis <[email protected]> Refs: https://github.com/nodejs/node-gyp/commit/91744bfecc67fda7db58e2a1c7aa72f196d6da4f PR-URL: https://github.com/nodejs/node/pull/28604 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: João Reis <[email protected]> * Revert "http: fix test where aborted should not be emitted" This reverts commit 461bf36d701f3f7c669e2d916d5a5bc17fc447bf. PR-URL: https://github.com/nodejs/node/pull/28699 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Matteo Collina <[email protected]> * doc: small grammar correction This commit improves the grammar of one sentence in the ESM documentation. PR-URL: https://github.com/nodejs/node/pull/28669 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: James M Snell <[email protected]> * inspector: do not change async call stack depth if the worker is done Fixes: https://github.com/nodejs/node/issues/28528 PR-URL: https://github.com/nodejs/node/pull/28613 Reviewed-By: Aleksei Koziatinskii <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: add missing version metadata for Readable.from Fixes: https://github.com/nodejs/node/issues/28693 PR-URL: https://github.com/nodejs/node/pull/28695 Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * doc: update js-native-api example Update example that shows how to separate N-API code which is not Node.js-specific from code which defines a Node.js N-API addon. In its existing state the example uses the pattern ```C assert(napi_*() == napi_ok); ``` However, this would result in no N-API calls when building with `-DNDEBUG`. This change moves away from assert and uses a macro `NAPI_CALL()` which throws the string corresponding to the non-`napi_ok` status as a JS exception and short-circuits the binding by returning `NULL`. PR-URL: https://github.com/nodejs/node/pull/28657 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: fix minor typo PR-URL: https://github.com/nodejs/node/pull/28148 Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: V8: backport d2ccc59 Original commit message: [snapshot] print reference stack for JSFunctions in the isolate snapshot This helps debugging incorrect usage of the SnapshotCreator API in debug mode. Change-Id: Ibd9db76a5f460cdf7ea6d14e865592ebaf69aeef Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1648240 Reviewed-by: Yang Guo <[email protected]> Commit-Queue: Yang Guo <[email protected]> Cr-Commit-Position: refs/heads/master@{#62095} Refs: https://github.com/v8/v8/commit/d2ccc599c7a31838752350ae927e41bc386df414 PR-URL: https://github.com/nodejs/node/pull/28648 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> * src: large pages option: FreeBSD support proposal Enabling on amd64 and as Linux, are 2MB large. The ELF section linkage script is compatible only with GNU ld. PR-URL: https://github.com/nodejs/node/pull/28331 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * module: increase code coverage of cjs loader Add test cases to cover uncovered wrap and wrapper getters. Refs: https://coverage.nodejs.org/coverage-99268b1e996d13a0/lib/internal/modules/cjs/loader.js.html#L153 PR-URL: https://github.com/nodejs/node/pull/27898 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jan Krems <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * test: use openssl_is_fips instead of hasFipsCrypto Currently, when dynamically linking against a FIPS enabled OpenSSL library test-process-env-allowed-flags-are-documented will fail with the following error: assert.js:89 throw new AssertionError(obj); ^ AssertionError [ERR_ASSERTION]: The following options are not documented as allowed in NODE_OPTIONS in /root/node/doc/api/cli.md: --enable-fips --force-fips at Object.<anonymous> (/test/parallel/test-process-env-allowed-flags-are-documented.js:82:8) at Module._compile (internal/modules/cjs/loader.js:779:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:790:10) at Module.load (internal/modules/cjs/loader.js:642:32) at Function.Module._load (internal/modules/cjs/loader.js:555:12) at Function.Module.runMain (internal/modules/cjs/loader.js:842:10) at internal/main/run_main_module.js:17:11 { generatedMessage: false, code: 'ERR_ASSERTION', actual: 2, expected: 0, operator: 'strictEqual' } This commit updates the test to use process.config.variables.openssl_is_fips instead of common.hasFipsCrypto as hasFipsCrypto only returns true if the OpenSSL library that is shipped with node was configured with FIPS enabled. PR-URL: https://github.com/nodejs/node/pull/28507 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * test: update hasFipsCrypto in test/common/README PR-URL: https://github.com/nodejs/node/pull/28507 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * http: expose headers on an http.ClientRequest "information" event 1xx intermediate status responses are allowed to have headers; so expose the "httpVersion", "httpVersionMajor", "httpVersionMinor", "headers", "rawHeaders", and "statusMessage" properties on this event. PR-URL: https://github.com/nodejs/node/pull/28459 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]> * readline: expose stream API in clearLine() This commit adds an optional callback to clearLine(), which is passed to the stream's write() method. It also exposes the return value of write(). PR-URL: https://github.com/nodejs/node/pull/28674 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * readline: expose stream API in moveCursor() This commit adds an optional callback to moveCursor(), which is passed to the stream's write() method. It also exposes the return value of write(). PR-URL: https://github.com/nodejs/node/pull/28674 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * readline: expose stream API in cursorTo() This commit adds an optional callback to cursorTo(), which is passed to the stream's write() method. It also exposes the return value of write(). PR-URL: https://github.com/nodejs/node/pull/28674 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * http: add response.writableFinished response.writableFinished is true if all data has been flushed to the underlying system. PR-URL: https://github.com/nodejs/node/pull/28681 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Rich Trott <[email protected]> * gyp: cherrypick more Python3 changes from node-gyp PR-URL: https://github.com/nodejs/node/pull/28563 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> * n-api: correct bug in napi_get_last_error napi_get_last_error returns incorrect napi_status. PR-URL: https://github.com/nodejs/node/pull/28702 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Gabriel Schulhof <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Michael Dawson <[email protected]> * build: specify Python version once for all tests Extracted from #28537 for shorter review cycle. This makes it easier to experiment with new versions of Python as they become available on the Travis CI platform. PR-URL: https://github.com/nodejs/node/pull/28694 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * test: improve variable names in pty_helper.py Using names like `parent_fd` and `child_fd` is more accurate here, and doesn’t come with unnecessary negative connotations, even if the previous naming is somewhat common terminology here. PR-URL: https://github.com/nodejs/node/pull/28688 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * test: fix race condition in test-worker-process-cwd.js This simplifies the test logic and fixes the race condition that could happen right now. PR-URL: https://github.com/nodejs/node/pull/28609 Refs: https://github.com/nodejs/node/issues/28193 Closes: https://github.com/nodejs/node/pull/28477 Fixes: https://github.com/nodejs/node/issues/27669 Fixes: https://github.com/nodejs/node/issues/28477 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> * test: make repl tests more resilient This refactors two tests to ignore line numbers in stack traces. That way changed line numbers do not have any impact on the test outcome anymore. PR-URL: https://github.com/nodejs/node/pull/28608 Fixes: https://github.com/nodejs/node/issues/28546 Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Lance Ball <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> * repl: fix autocomplete while using .load This makes sure that complete functions work as expected after using the REPL's `.load` command. It also fixes the corresponding test. So far the assertion where swallowed and the test passed even though it should not have. Fixes: https://github.com/nodejs/node/issues/28546 PR-URL: https://github.com/nodejs/node/pull/28608 Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Lance Ball <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> * repl: fix some repl context issues This partially fixes contexts like `{} instanceof Object === false` in the REPL. This does not fix all cases, since it's something fundamental from the REPL's design that things like these can happen. Refs: https://github.com/nodejs/node/issues/27859 PR-URL: https://github.com/nodejs/node/pull/28561 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: add examples at assert.strictEqual PR-URL: https://github.com/nodejs/node/pull/28092 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: improve os.homedir() docs PR-URL: https://github.com/nodejs/node/pull/28401 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: add example for zlib.createGzip() PR-URL: https://github.com/nodejs/node/pull/28136 Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Masashi Hirano <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: add example for beforeExit event PR-URL: https://github.com/nodejs/node/pull/28430 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * test: use consistent test naming To conform with other test names, move test/async-hooks/test-httparser-reuse.js to test/async-hooks/test-httpparser-reuse.js. PR-URL: https://github.com/nodejs/node/pull/28744 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * test: propagate napi_status to JS Re: https://github.com/nodejs/node/pull/27945#discussion_r288833979 This commit regards reporting to the JS level an actual event that happens when using suspected improper null arguments. It is better to report the exact reason from N-API to the JS level. PR-URL: https://github.com/nodejs/node/pull/28505 Reviewed-By: Gabriel Schulhof <[email protected]> Reviewed-By: Michael Dawson <[email protected]> Reviewed-By: Rich Trott <[email protected]> * tty: expose stream API from readline methods This commit exposes the return value and callback of the underlying readline APIs from the tty module. PR-URL: https://github.com/nodejs/node/pull/28721 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * test,win: cleanup exec-timeout processes When CMD is used to launch a process and CMD is killed too quickly, the process can stay behind running in suspended state, never completing. This only happens in Windows Server 2008R2. Refs: https://github.com/nodejs/build/issues/1829 PR-URL: https://github.com/nodejs/node/pull/28723 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Beth Griggs <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * esm: implement "pkg-exports" proposal Refs: https://github.com/jkrems/proposal-pkg-exports/issues/36 PR-URL: https://github.com/nodejs/node/pull/28568 Reviewed-By: Anna Henningsen <[email protected]> * deps: upgrade npm to 6.10.0 PR-URL: https://github.com/nodejs/node/pull/28525 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Roman Reiss <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Daijiro Wachi <[email protected]> * http: avoid extra listener PR-URL: https://github.com/nodejs/node/pull/28705 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Fedor Indutny <[email protected]> * vm: remove usage of public util module PR-URL: https://github.com/nodejs/node/pull/28460 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Masashi Hirano <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * zlib: remove usage of public util module PR-URL: https://github.com/nodejs/node/pull/28454 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: James M Snell <[email protected]> * build: fix building with d8 PR-URL: https://github.com/nodejs/node/pull/28733 Reviewed-By: Gus Caplan <[email protected]> Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * test: changed function to arrow function Convert callback functions that are anonymous to arrow functions for better readability. Also adjusted the `this` object in places where that was required. PR-URL: https://github.com/nodejs/node/pull/28726 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Masashi Hirano <[email protected]> Reviewed-By: Beth Griggs <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Rich Trott <[email protected]> * build: update of the large page option error Now large pages is also supported by FreeBSD. PR-URL: https://github.com/nodejs/node/pull/28729 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: V8: backport b33af60 Original commit message: [api] Get ScriptOrModule from CompileFunctionInContext Adds a new out param which allows accessing the ScriptOrModule of a function, which allows an embedder such as Node.js to use the function's i::Script lifetime. Refs: https://github.com/nodejs/node-v8/issues/111 Change-Id: I34346d94d76e8f9b8377c97d948673f4b95eb9d5 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1699698 Reviewed-by: Yang Guo <[email protected]> Commit-Queue: Yang Guo <[email protected]> Cr-Commit-Position: refs/heads/master@{#62830} Refs: https://github.com/v8/v8/commit/b33af60dd9e7e5b2557b9fbf3fdb80209f6db844 PR-URL: https://github.com/nodejs/node/pull/28671 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Guy Bedford <[email protected]> * vm: fix gc bug with modules and compiled functions PR-URL: https://github.com/nodejs/node/pull/28671 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Guy Bedford <[email protected]> * build: skip test-ci doc targets if no crypto PR-URL: https://github.com/nodejs/node/pull/28747 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Rich Trott <[email protected]> * lib: rename lib/internal/readline.js This commit moves lib/internal/readline.js to lib/internal/readline/utils.js. This is in preparation of adding a readline.promises implementation. PR-URL: https://github.com/nodejs/node/pull/28753 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Shingo Inoue <[email protected]> * doc: amplify warning for execute callback Add specific recommendation not to use the to the napi-env parameter in napi_async_execute_callback PR-URL: https://github.com/nodejs/node/pull/28738 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: add information for heap snapshot flag It's nice to have usage examples, especially since the flag requires the `SIG` version of the signal name, unlike `kill`. PR-URL: https://github.com/nodejs/node/pull/28754 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * doc: add code example to subprocess.stdout PR-URL: https://github.com/nodejs/node/pull/28402 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Rich Trott <[email protected]> * policy: add policy-integrity to mitigate policy tampering PR-URL: https://github.com/nodejs/node/pull/28734 Reviewed-By: Gus Caplan <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * inspector: do not spin-wait while waiting for the initial connection Fixes: https://github.com/nodejs/node/issues/28741 PR-URL: https://github.com/nodejs/node/pull/28756 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Aleksei Koziatinskii <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> * src: add public virtual destructor for KVStore As KVStore has derived classes, it is essential to declare a public virtual destructor in the base KVStore class. Otherwise, deleting derived class instances using base class pointers would potentially cause undefined behaviour. Additionally, since we are implementing a non-default destructor, the special member functions have also been implemented in order to abide by the rule of five. PR-URL: https://github.com/nodejs/node/pull/28737 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> * http2: compat req.complete PR-URL: https://github.com/nodejs/node/pull/28627 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Rich Trott <[email protected]> * inspector: add inspector.waitForDebugger() This method blocks current node process until a client sends Runtime.runifWaitingForDebugger. It can be useful when we need to report inspector.url() before waiting for connection: ``` inspector.open(0, undefined, false); fs.writeFileSync(someFileName, inspector.url()); inspector.waitForDebugger(); ``` PR-URL: https://github.com/nodejs/node/pull/28453 Reviewed-By: Eugene Ostroukhov <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> * stream: add null push transform in async_iterator when the readable side of a transform ends any for await loop on that transform stream should also complete. This fix prevents for await loop on a transform stream from hanging indefinitely. PR-URL: https://github.com/nodejs/node/pull/28566 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * test: fix assertion argument order in test-esm-namespace PR-URL: https://github.com/nodejs/node/pull/28474 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: expose TraceEventHelper with NODE_EXTERN As node requires a tracing controller to be initialized embedders need access to the TraceEventHelper so that we can actually set the tracing controller. Refs: https://github.com/electron/electron/commit/0e5b6f93000e4718c9e35332ddbd0f6b76cdd585/#diff-89b287b2edd0a02dddae60cb26157f47 PR-URL: https://github.com/nodejs/node/pull/28724 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: update nghttp2 to 1.39.1 PR-URL: https://github.com/nodejs/node/pull/28448 Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * doc: claim NODE_MODULE_VERSION=75 for Electron 7 PR-URL: https://github.com/nodejs/node/pull/28774 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> * src: silence compiler warning This commit fixes the following warning: warning: missing field 'exports' initializer PR-URL: https://github.com/nodejs/node/pull/28764 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> * doc: update env default on child_process functions PR-URL: https://github.com/nodejs/node/pull/28776 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * tools: remove unused pkgsrc directory The pkgsrc Makefile target was removed in 2015 Refs: https://github.com/nodejs/node/pull/1938 PR-URL: https://github.com/nodejs/node/pull/28783 Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: make `CompiledFnEntry` a `BaseObject` In particular: - Move the class definition to the relevant header file, i.e. `node_contextify.h`. - Make sure that class instances are destroyed on `Environment` teardown. - Make instances of the key object traceable in heap dumps. This is particularly relevant here because our C++ script → map key mapping could introduce memory leaks when the import function metadata refers back to the script in some way. Refs: https://github.com/nodejs/node/pull/28671 PR-URL: https://github.com/nodejs/node/pull/28782 Reviewed-By: Gus Caplan <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: do not include partial AsyncWrap instances in heap dump Heap dumps can be taken either through the inspector or the public API for it during an async_hooks init() hook, but at that point the AsyncWrap in question is not done initializing yet and virtual methods cannot be called on it. Address this issue (somewhat hackily) by excluding `AsyncWrap` instances which have not yet executed their `init()` hook fully from heap dumps. Fixes: https://github.com/nodejs/node/issues/28786 PR-URL: https://github.com/nodejs/node/pull/28789 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> * tools: update ESLint to 6.1.0 Update ESLint to 6.1.0 PR-URL: https://github.com/nodejs/node/pull/28793 Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> * dns: fix unsigned record values Fixes: https://github.com/nodejs/node/issues/28790 PR-URL: https://github.com/nodejs/node/pull/28792 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> * deps: float 15d7e79 from openssl The upstream commit fixes an incorrect initialization of memory in rand_lib.c. This fixes all errors that are reported by valgrind during startup. Origin: https://github.com/openssl/openssl/commit/15d7e7997e219fc PR-URL: https://github.com/nodejs/node/pull/28796 Fixes: https://github.com/nodejs/node/issues/28739 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Shigeki Ohtsu <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Sam Roberts <[email protected]> * src: fix type name in comment The comment refers to an exception type that JS land throws on the C++ code's behalf but apparently I changed the JS name before landing the pull request and forgot to update the comment. Refs: https://github.com/nodejs/node/pull/20816 PR-URL: https://github.com/nodejs/node/pull/28320 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> * 2019-07-23, Version 12.7.0 (Current) Notable changes: * deps: * Updated nghttp2 to 1.39.1. https://github.com/nodejs/node/pull/28448 * Updated npm to 6.10.0. https://github.com/nodejs/node/pull/28525 * esm: * Implemented experimental "pkg-exports" proposal. A new `"exports"` field can be added to a module's `package.json` file to provide custom subpath aliasing. See https://github.com/jkrems/proposal-pkg-exports/ for more information. https://github.com/nodejs/node/pull/28568 * http: * Added `response.writableFinished`. https://github.com/nodejs/node/pull/28681 * Exposed `headers`, `rawHeaders` and other fields on an `http.ClientRequest` `"information"` event. https://github.com/nodejs/node/pull/28459 * inspector: * Added `inspector.waitForDebugger()`. https://github.com/nodejs/node/pull/28453 * policy: * Added `--policy-integrity=sri` CLI option to mitigate policy tampering. If a policy integrity is specified and the policy does not have that integrity, Node.js will error prior to running any code. https://github.com/nodejs/node/pull/28734 * readline,tty: * Exposed stream API from various methods which write characters. https://github.com/nodejs/node/pull/28674 https://github.com/nodejs/node/pull/28721 * src: * Use cgroups to get memory limits. This improves the way we set the memory ceiling for a Node.js process. Previously we would use the physical memory size to estimate the necessary V8 heap sizes. The physical memory size is not necessarily the correct limit, e.g. if the process is running inside a docker container or is otherwise constrained. This change adds the ability to get a memory limit set by linux cgroups, which is used by docker containers to set resource constraints. https://docs.docker.com/config/containers/resource_constraints/ https://github.com/nodejs/node/pull/27508 PR-URL: https://github.com/nodejs/node/pull/28817 * lib: support min/max values in validateInteger() This commit updates validateInteger() in two ways: - Number.isInteger() is used instead of Number.isSafeInteger(). This ensures that all integer values are supported. - Minimum and maximum values are supported. They default to the min and max safe integer values, but can be customized. PR-URL: https://github.com/nodejs/node/pull/28810 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * module: implement "exports" proposal for CommonJS Refs: https://github.com/jkrems/proposal-pkg-exports/issues/36 Refs: https://github.com/nodejs/node/pull/28568 PR-URL: https://github.com/nodejs/node/pull/28759 Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: Bradley Farias <[email protected]> * doc: api/stream.md typo from writeable to writable PR-URL: https://github.com/nodejs/node/pull/28822 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Richard Lau <[email protected]> * crypto: increase maxmem range from 32 to 53 bits Fixes: https://github.com/nodejs/node/issues/28755 PR-URL: https://github.com/nodejs/node/pull/28799 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * tools: update certdata.txt This is the certdata.txt[0] from NSS 3.45, released on 2019-07-05. This is the version of NSS that will ship in Firefox 69 on 2019-09-03. [0] https://hg.mozilla.org/projects/nss/raw-file/NSS_3_45_RTM/lib/ckfw/builtins/certdata.txt PR-URL: https://github.com/nodejs/node/pull/28808 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * crypto: update root certificates Update the list of root certificates in src/node_root_certs.h with tools/mk-ca-bundle.pl. Certificates added: (none) Certificates removed: - Certinomis - Root CA PR-URL: https://github.com/nodejs/node/pull/28808 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * doc: fix type in NSS update instructions The perl script must be fully named, correct so that the command can be copy-pasted-run from the docs. PR-URL: https://github.com/nodejs/node/pull/28808 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * build: `uname -m` is amd64 on freebsd, not x86_64 Fixes: https://github.com/nodejs/node/issues/13150 PR-URL: https://github.com/nodejs/node/pull/28804 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src : elevate v8 namespaces Leverage `using` semantics for repeated usage of v8 artifacts. PR-URL: https://github.com/nodejs/node/pull/28801 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> * doc: add documentation for response.flushHeaders() PR-URL: https://github.com/nodejs/node/pull/28807 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: claim NODE_MODULE_VERSION=76 for Electron 8 PR-URL: https://github.com/nodejs/node/pull/28809 Refs: https://github.com/electron/electron/projects/20#card-24099810 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Michael Dawson <[email protected]> * deps: backport b107214 from upstream V8 Original commit message: [code-serializer] Handlify in CodeSerializer::Deserialize This section potentially contains allocations and thus gc, all object references should be handlified. Bug: v8:9333 Change-Id: I5814e66e8b9b75a8bd952afecae7a3a27b42a642 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1647695 Auto-Submit: Jakob Gruber <[email protected]> Commit-Queue: Simon Zünd <[email protected]> Reviewed-by: Simon Zünd <[email protected]> Cr-Commit-Position: refs/heads/master@{#62012} (This required resolution of a few merge conflicts, so it’s essentially a manual backport.) Refs: https://github.com/v8/v8/commit/b10721426503b87d013ecf314ca139fa5334ebb7 Refs: https://github.com/nodejs/node/pull/28847 PR-URL: https://github.com/nodejs/node/pull/28850 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Jan Krems <[email protected]> * domain: use strong reference to domain while active When an uncaught exception is thrown inside a domain, the domain is removed from the stack as of 43a51708589ac789ce08beaeb49d6d778dfbdc49. This means that it might not be kept alive as an object anymore, and may be garbage collected before the `after()` hook can run, which tries to exit it as well. Resolve that by making references to the domain strong while it is active. Fixes: https://github.com/nodejs/node/issues/28275 PR-URL: https://github.com/nodejs/node/pull/28313 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Rich Trott <[email protected]> * build: re-enable openssl arm for arm64 In #23913 it looked like arm64 testing was flaky, and as a result a number of changes were made including disabling openssl asm for that architecture. This PR is limited to re-enabling the one change to turn on openssl asm for arm64, in the hopes that either it works now, or that this specific change introduces a reproducible condition. See: https://github.com/nodejs/node/pull/23913 PR-URL: https://github.com/nodejs/node/pull/28180 Reviewed-By: Rod Vagg <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: describe why new Buffer() is problematic Existing docs weren't clear on the actual problem. In addition, the text described 8.0.0 as being a future Node.js release, so adjust language to reflect that 8.0.0 is in the past (while not losing important information about what the pre-8.x behaviour was). PR-URL: https://github.com/nodejs/node/pull/28825 Reviewed-By: Michael Dawson <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * n-api: add APIs for per-instance state management Adds `napi_set_instance_data()` and `napi_get_instance_data()`, which allow native addons to store their data on and retrieve their data from `napi_env`. `napi_set_instance_data()` accepts a finalizer which is called when the `node::Environment()` is destroyed. This entails rendering the `napi_env` local to each add-on. Fixes: https://github.com/nodejs/abi-stable-node/issues/378 PR-URL: https://github.com/nodejs/node/pull/28682 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Michael Dawson <[email protected]> * doc: fix incorrect name in report docs In diagnostic reports, the CPUs are listed in a "cpus" field. This commit fixes the docs, which refer to the field as "osCpus" PR-URL: https://github.com/nodejs/node/pull/28830 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * report: loop over uv_cpu_info() results The code currently loops over the results, but only the first result is accessed. PR-URL: https://github.com/nodejs/node/pull/28829 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * assert: avoid potentially misleading reference to object identity Often, the word “identical” when referring to JS objects will be read as referring to having the same object identity (which is called “reference equality” here), but what the error message is trying to say here is that the objects are different but yield the same `util.inspect()` output. Since `util.inspect()` output represents the structure rather than the identity of objects, (hopefully) clarify the error message to reflect that. PR-URL: https://github.com/nodejs/node/pull/28824 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * crypto: add outputLength option to crypto.createHash This change adds an outputLength option to crypto.createHash which allows users to produce variable-length hash values using XOF hash functons. Fixes: https://github.com/nodejs/node/issues/28757 PR-URL: https://github.com/nodejs/node/pull/28805 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Rich Trott <[email protected]> * crypto: fix handling of malicious getters (scrypt) It is possible to bypass parameter validation in crypto.scrypt and crypto.scryptSync by crafting option objects with malicious getters as demonstrated in the regression test. After bypassing validation, any value can be passed to the C++ layer, causing an assertion to crash the process. Fixes: https://github.com/nodejs/node/issues/28836 PR-URL: https://github.com/nodejs/node/pull/28838 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * policy: add dependencies map for resources Adds a "dependencies" field to resources in policy manifest files. In order to ease development and testing while using manifests, wildcard values for both "dependencies" and "integrity" have been added using the boolean value "true" in the policy manifest. PR-URL: https://github.com/nodejs/node/pull/28767 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Jan Krems <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: add documentation for stream.destroyed PR-URL: https://github.com/nodejs/node/pull/28815 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Matteo Collina <[email protected]> * n-api: refactor a previous commit This is a refactoring of https://github.com/nodejs/node/issues/27628 following https://github.com/nodejs/node/pull/28505. This change factors out functions `add_last_status()` and `add_returned_status()` so they may be reused in the tests for passing information about the last error status and/or a returned `napi_status` to JavaScript. PR-URL: https://github.com/nodejs/node/pull/28848 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Gabriel Schulhof <[email protected]> * stream: resolve perf regression introduced by V8 7.3 This commit contains two fixes: 1. use instanceof instead of Object.getPrototypeOf, as checking an object prototype with Object.getPrototypeOf is slower than an instanceof check. 2. avoid parseInt(undefined, 10) to get NaN as it regressed. PR-URL: https://github.com/nodejs/node/pull/28842 Fixes: https://github.com/nodejs/node/issues/28586 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * build,tools: support building with Visual Studio 2019 Add a `vs2019` option to `vcbuild.bat` to use VS 2019 instead of VS 2017 PR-URL: https://github.com/nodejs/node/pull/28781 Reviewed-By: João Reis <[email protected]> Reviewed-By: Refael Ackermann (רפאל פלחי) <[email protected]> * test: specialize OOM check for AIX Assumption that if memory can be malloc()ed it can be used is not true on AIX. Later access of the allocated pages can trigger SIGKILL if there are insufficient VM pages. Use psdanger() to better estimate available memory. Fixes: https://github.com/nodejs/build/issues/1849 More info: - https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/generalprogramming/sys_mem_alloc.html - https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/p_bostechref/psdanger.html Related to: - https://github.com/nodejs/build/issues/1820#issuecomment-505998851 - https://github.com/nodejs/node/pull/28469 - https://github.com/nodejs/node/pull/28516 PR-URL: https://github.com/nodejs/node/pull/28857 Reviewed-By: Michael Dawson <[email protected]> Reviewed-By: Beth Griggs <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: move relative uptime init PR-URL: https://github.com/nodejs/node/pull/28849 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * http: reset parser.incoming when server response is finished This resolves a memory leak for keep-alive connections with a naïve approach. Fixes: https://github.com/nodejs/node/issues/9668 PR-URL: https://github.com/nodejs/node/pull/28646 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: remove backup files PR-URL: https://github.com/nodejs/node/pull/28865 Reviewed-By: Richard Lau <riclau@uk.…

addaleax · 2019-08-15T18:08:17Z

@tniessen Could you backport this to v10.x?

michaelsbradleyjr · 2019-08-15T23:16:04Z

@tniessen I was working today on a refactor of the shim I mentioned in #28755. In a Node REPL of either v12.8.0 or v12.8.1 I'm still having a problem with some standard test data:

> require('crypto').scryptSync(Buffer.from('testpassword'), Buffer.from('ab0c7876052600dd703518d6fc3fe8984592145b591fc8fb5c6d43190334ba19', 'hex'), 32, {N: 262144, r: 1, p: 8, maxmem: 2 ** 53 - 1})
Thrown:
Error: error:060B50AC:digital envelope routines:EVP_PBE_scrypt:memory limit exceeded
    at Object.scryptSync (internal/crypto/scrypt.js:55:15)

I was excited when your changes landed in 12.8.0 but it wasn't until today that I had occasion to try them out relative to the problem that prompted my feature request.

Is this something that can be revisited?

tniessen · 2019-08-16T22:14:44Z

@addaleax Will do :)

@michaelsbradleyjr You are hitting this branch in OpenSSL:

if (16 * r <= LOG2_UINT64_MAX) {
    if (N >= (((uint64_t)1) << (16 * r))) {
        EVPerr(EVP_F_EVP_PBE_SCRYPT, EVP_R_MEMORY_LIMIT_EXCEEDED);
        return 0;
    }
}

RFC7914, section 2 specifies that "N [...] must be less than 2^(128 * r / 8)". So based on the RFC and on the OpenSSL implementation, I think your test case is invalid?

michaelsbradleyjr · 2019-08-16T22:49:21Z

@tniessen thanks for the clarification. I'm guessing that when adopting scrypt, the folks implementing it for go-ethereum and related tools didn't look to the RFC but to earlier work on the kdf:

The parameter N must be a power of 2 greater than 1.

That's just a guess, though; support for larger N could have been an accident or motivated by something else.

I suspect it's going to turn out to be a bit of a pain for the JS devs in the Ethereum community because there's an expectation that across runtimes the tools should conform to one another (and I wouldn't expect or request Node's maintainers to not follow the RFC for scrypt). It probably means one of the Ethereum-related foundations will need to in-house Barry Steyn's deprecated scrypt package and keep it up-to-date with ABI changes across Node.js releases... very do-able, just not quite as nice as a runtime built-in.

In any case, thank you for your time. While I've been using Node since its beginnings, I don't know a lot about cryptography internals (didn't even realize there is an RFC for scrypt) but am learning. Next time around I'll hunt first for clues in RFCs and the like, and I appreciate your pointing it out.

For others who end up here trying to figure out why Node's crypto.scrypt isn't "working" in some cases, here's a standalone shim that will fall back to a pure JS implementation:

https://github.com/web3-js/scrypt-shim

In time it may evolve to include a message that a not-deprecated addon is available for installation.

tniessen · 2019-08-17T08:45:44Z

@michaelsbradleyjr No problem, I am glad I could help! Feedback is always welcome :-)

earlier work on the kdf:

The parameter N * must be a power of 2 greater than 1.

I think the explanation on the front page is an oversimplification. I had a brief look at the original paper on the same website (from the same author) and managed to arrive at the same limit for N as the RFC:

The algorithm ROMix limits N to < 2^(k/8) (section 5), where k is the length of the output produced by BlockMix (section 6), so k = 2 * r * x, where x is the output length of Salsa20/8, so x = 64, yielding the limit 2^(k / 8) = 2^(2 * x * r / 8) = 2^(128 * r / 8). Which is precisely the limit specified in RFC7914.

I only had a brief look so I might have oversimplified this myself, but it seems that your test case violates both the RFC and the original paper.

Anyway, I don't think there is much we (or OpenSSL) could do here (apart from changing the error code to something more useful). Your solution sounds reasonable assuming you need to permit these invalid parameters. :-)

michaelsbradleyjr · 2019-08-18T12:40:30Z

@tniessen thanks for the follow-up. I'm going to do some additional research; at present, again for the sake of those who end up here trying to understand the situation, I'd summarize as follows:

Let scrypt* be a kdf whereby for every N, r, p combination that's valid for an RFC-compliant implementation of scrypt the same output will be produced by scrypt*, but scrypt* has different rules regarding what N, r, p combinations are valid. For example: {N: 262144, r: 1, p: 8} is valid for scrypt* but not for RFC-compliant scrypt because of the latter's rule that N < 2^(128 * r / 8) must be true.

What I'm going to research:

Across implementations of Ethereum and related tools, is there a consistent set of rules that can be used to exactly spec what I've termed scrypt*?
Was/is non-compliance with the RFC and original paper a deliberate decision or an accident?
If it was an accident has "the ship sailed" (probably) or could the Ethereum community (via the EIP process) officially adopt RFC-compliant scrypt?

If comments here are closed before I find the answers, they will eventually be included in the README of scrypt-shim.

It's not quite apples-to-apples, but this situation bears resemblance to "SHA-3" in the context of Ethereum referring to something different than the standard, though awareness of the difference has grown among Ethereum devs and many now use the term "keccak" instead of SHA-3 to avoid confusion.

tniessen · 2019-08-18T14:58:09Z

If comments here are closed before I find the answers

We only disable comments in very rare circumstances, so feel free to comment here if it is relevant to the discussion.

Fixes: nodejs#28755 PR-URL: nodejs#28799 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]>

Fixes: #28755 Backport-PR-URL: #29316 PR-URL: #28799 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]>

Notable changes: * crypto: * add support for chacha20-poly1305 for AEAD (chux0519) #24081 * increase maxmem range from 32 to 53 bits (Tobias Nießen) #28799 * deps: * update npm to 6.11.3 (claudiahdz) #29430 * upgrade openssl sources to 1.1.1d (Sam Roberts) #29921 * dns: * remove dns.promises experimental warning (cjihrig) #26592 * fs: * remove experimental warning for fs.promises (Anna Henningsen) #26581 * http: * makes response.writeHead return the response (Mark S. Everitt) #25974 * http2: * makes response.writeHead return the response (Mark S. Everitt) #25974 * n-api: * make func argument of napi\_create\_threadsafe\_function optional (legendecas) #27791 * mark version 5 N-APIs as stable (Gabriel Schulhof) #29401 * implement date object (Jarrod Connolly) #25917 * process: * add --unhandled-rejections flag (Ruben Bridgewater) #26599 * stream: * implement Readable.from async iterator utility (Guy Bedford) #27660 * make Symbol.asyncIterator support stable (Matteo Collina) #26989 PR-URL: #29875

tniessen · 2020-04-27T12:24:39Z

My original analysis was wrong. A few months ago, I found a mistake (incorrect conversion between bits/bytes) in the RFC and filed errata at the beginning of February (5971, 5972, and 5973). If someone fixes OpenSSL accordingly, Node.js will allow previously invalid inputs.

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. crypto Issues and PRs related to the crypto subsystem. labels Jul 21, 2019

tniessen mentioned this pull request Jul 21, 2019

Can the max value for the maxmem option of crypto.scrypt/Sync be reconsidered? #28755

Closed

bnoordhuis approved these changes Jul 21, 2019

View reviewed changes

lib/internal/crypto/scrypt.js Outdated Show resolved Hide resolved

addaleax approved these changes Jul 21, 2019

View reviewed changes

michaelsbradleyjr mentioned this pull request Jul 21, 2019

WIP: Remove scrypt.js dependency (help wanted) ethereumjs/ethereumjs-wallet#91

Closed

cjihrig approved these changes Jul 22, 2019

View reviewed changes

doc/api/crypto.md Outdated Show resolved Hide resolved

doc/api/crypto.md Outdated Show resolved Hide resolved

lib/internal/crypto/scrypt.js Show resolved Hide resolved

lpinca approved these changes Jul 22, 2019

View reviewed changes

tniessen added 2 commits July 22, 2019 23:35

crypto: increase maxmem range from 32 to 53 bits

ef735d6

Fixes: nodejs#28755

fixup! crypto: increase maxmem range from 32 to 53 bits

c37e0fc

tniessen force-pushed the crypto-scrypt-increase-maxmem-range branch from 0bee128 to c37e0fc Compare July 22, 2019 21:38

fixup! crypto: increase maxmem range from 32 to 53 bits

e9cb4ec

tniessen added the blocked PRs that are blocked by other issues or PRs. label Jul 23, 2019

cjihrig mentioned this pull request Jul 23, 2019

lib: support min/max values in validateInteger() #28810

Closed

2 tasks

Trott approved these changes Jul 23, 2019

View reviewed changes

Trott removed the blocked PRs that are blocked by other issues or PRs. label Jul 23, 2019

This comment has been minimized.

Sign in to view

fixup! crypto: increase maxmem range from 32 to 53 bits

2f5401a

Trott closed this Jul 24, 2019

michaelsbradleyjr mentioned this pull request Jul 25, 2019

Node >= v8.0.0 support for web3 1.0 web3/web3.js#2938

Merged

addaleax added lts-watch-v10.x and removed lts-watch-v10.x labels Aug 15, 2019

addaleax mentioned this pull request Aug 15, 2019

V10.17.0 proposal #29150

Closed

michaelsbradleyjr mentioned this pull request Aug 18, 2019

scrypt usage not compliant with RFC 7914? ethereum/go-ethereum#19977

Closed

michaelsbradleyjr mentioned this pull request Aug 18, 2019

scrypt example not compliant with RFC 7914 ethereum/wiki#674

Open

tniessen mentioned this pull request Aug 25, 2019

(v10.x backport) crypto: increase maxmem range from 32 to 53 bits #29316

Closed

michaelsbradleyjr mentioned this pull request Sep 25, 2019

x/crypto/scrypt: implementation not compliant with RFC 7914? golang/go#33703

Open

BethGriggs mentioned this pull request Oct 7, 2019

v10.17.0 proposal #29875

Merged

michaelsbradleyjr mentioned this pull request Dec 20, 2019

update scrypt shim using maxmem fix from node >= 12.8 web3/web3.js#3284

Closed

cgewecke mentioned this pull request May 4, 2020

Update ethereumjs-util to v7.0.1 ethereumjs/ethereumjs-wallet#121

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

crypto: increase maxmem range from 32 to 53 bits #28799

crypto: increase maxmem range from 32 to 53 bits #28799

tniessen commented Jul 21, 2019

bnoordhuis left a comment

nodejs-github-bot commented Jul 22, 2019

tniessen commented Jul 23, 2019

Trott commented Jul 23, 2019 •

edited

Loading

tniessen commented Jul 23, 2019

Trott commented Jul 23, 2019

Trott commented Jul 23, 2019

This comment has been minimized.

nodejs-github-bot commented Jul 23, 2019

Trott commented Jul 24, 2019

addaleax commented Aug 15, 2019

michaelsbradleyjr commented Aug 15, 2019 •

edited

Loading

tniessen commented Aug 16, 2019

michaelsbradleyjr commented Aug 16, 2019 •

edited

Loading

tniessen commented Aug 17, 2019

michaelsbradleyjr commented Aug 18, 2019 •

edited

Loading

tniessen commented Aug 18, 2019

tniessen commented Apr 27, 2020

crypto: increase maxmem range from 32 to 53 bits #28799

crypto: increase maxmem range from 32 to 53 bits #28799

Conversation

tniessen commented Jul 21, 2019

Checklist

bnoordhuis left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Jul 22, 2019

tniessen commented Jul 23, 2019

Trott commented Jul 23, 2019 • edited Loading

tniessen commented Jul 23, 2019

Trott commented Jul 23, 2019

Trott commented Jul 23, 2019

This comment has been minimized.

nodejs-github-bot commented Jul 23, 2019

Trott commented Jul 24, 2019

addaleax commented Aug 15, 2019

michaelsbradleyjr commented Aug 15, 2019 • edited Loading

tniessen commented Aug 16, 2019

michaelsbradleyjr commented Aug 16, 2019 • edited Loading

tniessen commented Aug 17, 2019

michaelsbradleyjr commented Aug 18, 2019 • edited Loading

tniessen commented Aug 18, 2019

tniessen commented Apr 27, 2020

Trott commented Jul 23, 2019 •

edited

Loading

michaelsbradleyjr commented Aug 15, 2019 •

edited

Loading

michaelsbradleyjr commented Aug 16, 2019 •

edited

Loading

michaelsbradleyjr commented Aug 18, 2019 •

edited

Loading