-
Notifications
You must be signed in to change notification settings - Fork 464
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
enable unit tests #1078
enable unit tests #1078
Conversation
@mhdawson @KevinEady @gabrielschulhof PR is ready for review |
a43bd0c
to
7190df5
Compare
package.json
Outdated
@@ -378,7 +378,9 @@ | |||
"dev:incremental": "node test", | |||
"doc": "doxygen doc/Doxyfile", | |||
"lint": "eslint $(git diff --name-only refs/remotes/origin/main '**/*.js' | xargs) && node tools/clang-format", | |||
"lint:fix": "node tools/clang-format --fix && eslint --fix $(git diff --cached --name-only '**/*.js' | xargs && git diff --name-only '**/*.js' | xargs)" | |||
"lint:fix": "node tools/clang-format --fix && eslint --fix $(git diff --cached --name-only '**/*.js' | xargs && git diff --name-only '**/*.js' | xargs)", | |||
"preunit": "filter=\"$npm_config_filter\" node-gyp rebuild -C unit-test", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you need to copy $npm_config_filter into $filter? You can process $npm_config_filter directly in binding.gyp.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielschulhof , I wanted to refer to the passed in argument "filter" by it's original name, because npm translates it into npm_config_filter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK. I understand. Still, I think for the simplicity of the command line it would be better to move this assignment into the JS file. So, like
// npm prepends "npm_config_" to all environment variables created from npm command line parameters.
let filter = process.env.npm_config_filter;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can drop "filter=\"$npm_config_filter\"
because you're using process.env.npm_config_filter
inside the script now.
test/index.js
Outdated
@@ -29,9 +29,23 @@ if (typeof global.gc !== 'function') { | |||
|
|||
const fs = require('fs'); | |||
const path = require('path'); | |||
process.env.filter = require('../unit-test/matchModules').matchWildCards(process.env.filter); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could easily be process.env.npm_config_filter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielschulhof IMO using npm_config_filter seems slightly misleading, that is why I translated to "filter" , But I am open to changing this
unit-test/.gitignore
Outdated
@@ -0,0 +1,3 @@ | |||
/node_modules | |||
/build | |||
/generated |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please make sure there are newlines at the ends of files!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielschulhof agreed
unit-test/binding.gyp
Outdated
'dependencies': [ 'generateBindingCC' ] | ||
}, | ||
], | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a newline!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielschulhof thanks, agreed
unit-test/binding.gyp
Outdated
@@ -0,0 +1,64 @@ | |||
{ | |||
'target_defaults': { | |||
'includes': ['common.gypi'], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not ../common.gypi and then you don't need to copy the file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielschulhof agreed
unit-test/generate-binding-cc.js
Outdated
* | ||
* Test cases | ||
* @fires only when run directly from terminal with TEST=true | ||
* | ||
* | ||
* | ||
* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need the blank lines surrounding the comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed
unit-test/injectTestParams.js
Outdated
* | ||
* @returns : list of files to compile by node-gyp | ||
* @param : none | ||
* @requires : picks `filter` parameter from process.env | ||
* This function is used as an utility method to inject a list of files to compile into binding.gyp | ||
* | ||
* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here. Do we need the first and trailing blank lines?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielschulhof agreed , removing blank lines
unit-test/injectTestParams.js
Outdated
* @param : none | ||
* @requires : picks `filter` parameter from process.env | ||
* This function is used as an utility method by the generateBindingCC step in binding.gyp | ||
* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Blank line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed
unit-test/injectTestParams.js
Outdated
* | ||
* | ||
* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need these blank lines? Here and in all these comments?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed
unit-test/spawnTask.js
Outdated
/* | ||
* spawns a child process to run a given node.js script | ||
*/ | ||
module.exports.runChildProcess = async function (scriptName, options) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The function doesn't need to be async if it returns a promise.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It can be used from an async function without itself being marked as async.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielschulhof agreed
4bc30b5
to
defc460
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See comments.
test/index.js
Outdated
@@ -87,7 +108,7 @@ if (napiVersion < 3) { | |||
testModules.splice(testModules.indexOf('version_management'), 1); | |||
} | |||
|
|||
if (napiVersion < 4) { | |||
if (napiVersion < 4 && !filterConditionFiles.length) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to check for !filterConditionFiles.length
here (and in the lines below as well)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@KevinEady I am using the array filterConditionFiles
as a flag to, Not apply versioning for filtered tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@KevinEady suppose I specifically run tests for only 'asyncprogressqueueworker', I would expect the tests to run even if it does not satisfy node version criteria (the tests would fail which is ok, I can upgrade local node version and try again)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't treat .length
as a boolean!
threadsafe: 'threadSafe', | ||
objectwrap: 'objectWrap' | ||
}, | ||
exportNames: { | ||
AsyncWorkerPersistent: 'PersistentAsyncWorker' | ||
}, | ||
propertyNames: { | ||
async_worker_persistent: 'persistentasyncworker', | ||
objectwrap_constructor_exception: 'objectwrapConstructorException' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did we discuss standardizing the test files themselves so this sort of manual checks aren't needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@KevinEady in some cases the exception from the standard form seems to be valid,
eg: persistentasyncworker
is a good name to export from the test file async_worker_persistent
const workingDir = path.join(__dirname, '../'); | ||
const relativeBuildPath = path.join('../', 'unit-test'); | ||
const buildPath = path.join(__dirname, './unit-test'); | ||
const envVars = { ...process.env, REL_BUILD_PATH: relativeBuildPath, BUILD_PATH: buildPath }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it needed to have both REL_BUILD_PATH
and BUILD_PATH
...?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@KevinEady yes, the relative build path is required in a specific case in test/index.js where it determines the target build configuration (DEBUG or RELEASE)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the build path is necessary to import the test modules and run tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am still a little confused as to why you can't you also use the relative path instead of introducing more variables?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@KevinEady in
Line 93 in 1f160cf
.readdirSync(path.join(__dirname, process.env.REL_BUILD_PATH || '', 'build')) |
-----> ./build
test ----
-----> ../unit/ -> /build
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that path reference is different from the path reference in
node-addon-api/test/common/index.js
Line 78 in 1f160cf
exports.runTest = async function (test, buildType, buildPathRoot = process.env.BUILD_PATH || '') { |
where we need the complete root to a build path
unit-test/binding-file-template.js
Outdated
// content.push("Object InitName(Env env);"); | ||
inits.forEach(init => content.push(init)); | ||
|
||
content.push('Object Init(Env env, Object exports) {'); | ||
|
||
// content.push("exports.Set(\"name\", InitName(env));"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove unnecessary comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed
unit-test/generate-binding-cc.js
Outdated
generateBindingConfigurations().then(generateFileContent).then(writeToBindingFile); | ||
|
||
/** | ||
* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also at the end of this comment as well
29068da
to
2d70c10
Compare
@gabrielschulhof @KevinEady I have responded to all comments, please let me know your feedback |
@gabrielschulhof @KevinEady do you want to have another look before this lands? |
package.json
Outdated
@@ -378,7 +378,9 @@ | |||
"dev:incremental": "node test", | |||
"doc": "doxygen doc/Doxyfile", | |||
"lint": "eslint $(git diff --name-only refs/remotes/origin/main '**/*.js' | xargs) && node tools/clang-format", | |||
"lint:fix": "node tools/clang-format --fix && eslint --fix $(git diff --cached --name-only '**/*.js' | xargs && git diff --name-only '**/*.js' | xargs)" | |||
"lint:fix": "node tools/clang-format --fix && eslint --fix $(git diff --cached --name-only '**/*.js' | xargs && git diff --name-only '**/*.js' | xargs)", | |||
"preunit": "filter=\"$npm_config_filter\" node-gyp rebuild -C unit-test", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK. I understand. Still, I think for the simplicity of the command line it would be better to move this assignment into the JS file. So, like
// npm prepends "npm_config_" to all environment variables created from npm command line parameters.
let filter = process.env.npm_config_filter;
exports.mustNotCall = function(msg) { | ||
return function mustNotCall() { | ||
exports.mustNotCall = function (msg) { | ||
return function mustNotCall () { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a lot of whitespace changes that inflate the diff. Are they the result of running a linter?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it's the result of running the linter ☺
@gabrielschulhof I have changed the npm_config_filter refererence now. |
So, some of the addons don't run correctly in unit-test mode. I think we can tackle that at a later date. However, I think there is a bigger issue in that a failed unit test exits the
I think this is a blocker for merging the PR, as the return value should signify if the test passes or not. |
@deepakrkris please also remove the aliasing of npm_config_filter to filter from the command lines in package.json. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Marking as Request changes
per @KevinEady's comment about returning success even on error. Please dismiss when the problem is fixed!
@gabrielschulhof I have fixed this now |
@KevinEady @gabrielschulhof I have fixed this now |
@KevinEady I am able verify the fix as below
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! We should figure out why individual unit tests fail, but I think that can come at a different time.
I have verified the return code is now non-zero for failed unit tests:
➜ node-addon-api git:(bc7622e) ✗ npm run unit --filter=addon ; echo "Return code: $?"
> [email protected] preunit /Users/kevineady/Documents/Projects/node-addon-api
> node-gyp rebuild -C unit-test
ACTION Generating binding cc file generated/binding.cc
test modules to bind: [ 'addon' ]
generated binding file /Users/kevineady/Documents/Projects/node-addon-api/unit-test/generated/binding.cc 2021-12-10T16:03:38.620Z
TOUCH Release/obj.target/generateBindingCC.stamp
CXX(target) Release/obj.target/binding/generated/binding.o
CXX(target) Release/obj.target/binding/../test/addon.o
SOLINK_MODULE(target) Release/binding.node
CXX(target) Release/obj.target/binding_noexcept/generated/binding.o
CXX(target) Release/obj.target/binding_noexcept/../test/addon.o
SOLINK_MODULE(target) Release/binding_noexcept.node
CXX(target) Release/obj.target/binding_noexcept_maybe/generated/binding.o
CXX(target) Release/obj.target/binding_noexcept_maybe/../test/addon.o
SOLINK_MODULE(target) Release/binding_noexcept_maybe.node
CXX(target) Release/obj.target/binding_swallowexcept/generated/binding.o
CXX(target) Release/obj.target/binding_swallowexcept/../test/addon.o
SOLINK_MODULE(target) Release/binding_swallowexcept.node
CXX(target) Release/obj.target/binding_swallowexcept_noexcept/generated/binding.o
CXX(target) Release/obj.target/binding_swallowexcept_noexcept/../test/addon.o
SOLINK_MODULE(target) Release/binding_swallowexcept_noexcept.node
> [email protected] unit /Users/kevineady/Documents/Projects/node-addon-api
> node unit-test/test
Starting to run tests in /Users/kevineady/Documents/Projects/node-addon-api/unit-test/unit-test 2021-12-10T16:03:49.692Z
napiVersion:5
Testing with Node-API Version '5'.
Starting test suite
[ 'addon' ]
Running test 'addon'
error: dyld: lazy symbol binding failed: Symbol not found: __Z9InitAddonN4Napi3EnvE
Referenced from: /Users/kevineady/Documents/Projects/node-addon-api/unit-test/build/Release/binding.node
Expected in: flat namespace
dyld: Symbol not found: __Z9InitAddonN4Napi3EnvE
Referenced from: /Users/kevineady/Documents/Projects/node-addon-api/unit-test/build/Release/binding.node
Expected in: flat namespace
error: Tests aborted with SIGABRT
child process exited with code 1
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] unit: `node unit-test/test`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] unit script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/kevineady/.npm/_logs/2021-12-10T16_03_50_458Z-debug.log
Return code: 1
@KevinEady I cloned the repo (from my fork) freshly and the unit test with filter=addon runs good
|
@KevinEady I cloned the forked branch and worked on it. Is there another way you have cloned the repo like say cloning the root repo and then checking out to a forked branch ? I can try that as well. |
test/index.js
Outdated
let filterCondition = process.env.npm_config_filter || ''; | ||
let filterConditionFiles = []; | ||
|
||
if (process.env.npm_config_filter !== null && process.env.npm_config_filter !== undefined && process.env.npm_config_filter.trim() !== '') { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not if (filterCondition != null && ...)
, accounting for the fact that the default is ''
and not null
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed
test/index.js
Outdated
|
||
if (process.env.npm_config_filter !== null && process.env.npm_config_filter !== undefined && process.env.npm_config_filter.trim() !== '') { | ||
filterCondition = require('../unit-test/matchModules').matchWildCards(process.env.npm_config_filter); | ||
filterConditionFiles = filterCondition.split(' ').length ? filterCondition.split(' ') : [filterCondition]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should make this check explicitly > 0
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed
test/index.js
Outdated
function checkFilterCondition (fileName, parsedFilepath) { | ||
let result = false; | ||
|
||
if (!filterConditionFiles.length) result = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here (> 0
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed
test/index.js
Outdated
@@ -87,7 +108,7 @@ if (napiVersion < 3) { | |||
testModules.splice(testModules.indexOf('version_management'), 1); | |||
} | |||
|
|||
if (napiVersion < 4) { | |||
if (napiVersion < 4 && !filterConditionFiles.length) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't treat .length
as a boolean!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@deepakrkris I see there are a few final comments from @gabrielschulhof . If you can address those I'll land. |
@deepakrkris if you can squash down to 1 commit that would also make it much easier for me to land and it seems individual commits have conflicts. |
@deepakrkris ran the ci jobs https://ci.nodejs.org/job/node-test-node-addon-api-new/ for latest branch and 12.x and all seems ok so we should be good once you update the last few comments. If you can email me once you do that that will help me get to sooner. |
@mhdawson sure, I will fix the comments and ping here in a few hours, thank you |
8dcdcc6
to
6603f37
Compare
6603f37
to
17aaeca
Compare
@mhdawson I have fixed comments from Gabriel and squashed the commits |
#1078 (comment) , @gabrielschulhof agreed, fixed the comments now |
Landed as 744c8d2 |
Document all possible filter conditions added in nodejs#1078
* Update Readme for filter conditions in unit tests Document all possible filter conditions added in #1078 PR-URL:#1199 Reviewed-By: Michael Dawson <[email protected]>
* Update Readme for filter conditions in unit tests Document all possible filter conditions added in nodejs/node-addon-api#1078 PR-URL:nodejs/node-addon-api#1199 Reviewed-By: Michael Dawson <[email protected]>
* Update Readme for filter conditions in unit tests Document all possible filter conditions added in nodejs/node-addon-api#1078 PR-URL:nodejs/node-addon-api#1199 Reviewed-By: Michael Dawson <[email protected]>
* Update Readme for filter conditions in unit tests Document all possible filter conditions added in nodejs/node-addon-api#1078 PR-URL:nodejs/node-addon-api#1199 Reviewed-By: Michael Dawson <[email protected]>
* Update Readme for filter conditions in unit tests Document all possible filter conditions added in nodejs/node-addon-api#1078 PR-URL:nodejs/node-addon-api#1199 Reviewed-By: Michael Dawson <[email protected]>
Enable running tests with specific filter conditions:
Example:
Wildcards are also possible:
Example:
Multiple filter conditions are also allowed
Example: