Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test] Remove unit test case in zip.test.js #540

Merged
merged 1 commit into from
Jun 29, 2021

Conversation

ananzh
Copy link
Member

@ananzh ananzh commented Jun 24, 2021

Description

With existing fixture, test verify consistency of modes of files inside zip.test.js fails. This is due to diff file mode in different system when doing same command or operation. For example, in our ubuntu, when create a file using mkdir cmd, the file mode is 775. But local machine, the file mode is 755.

local env: drwxr-xr-x  62 ananzh  staff   1.9K Jun 23 19:12 OpenSearch-Dashboards (755)
Ubuntu: drwxrwxr-x 24 anan anan 4096 Jun 24 01:04 OpenSearch-Dashboards (775)

After unzip executable has mode 777 and not-executable has mode 644. These two numbers are fixed. They are not dependent on diff system or folder mode where you unzip the files. However, the following two results:

getMode(path.resolve(tempPath,'executable'))
getMode(path.resolve(tempPath,’not-executable'))

are dependent on tempPath mode because it will use the lower mode. For example (use mode# directly below):
1)If tempPath is 775 then:

getMode(path.resolve(775, 777)) —> 775
getMode(path.resolve(775, 644)) —> 644

2)If tempPath is 755 then:

getMode(path.resolve(755, 777)) —> 755
getMode(path.resolve(755, 644)) —> 644

To verify by yourself, you could change getMode function and for example pass it a 755:
getMode = (path) => (fs.statSync(path).mode & parseInt('755’, 8)).toString(8);
Then in the test case, console log the mode of tempPath and test result:

console.log(getMode(tempPath));
console.log(getMode(path.resolve(tempPath, 'executable')));

Therefore, this unit test case result is dependent on the mode of tempPath which is affected by your system or system settings.

This PR investigats the issue and remove todo. We have two options: 1) Our Jenkins test env has 755 for tempPath (see below image). So we could enable this test case without any changes. Customer might fail this test locally and can manually skip this test.

Screen Shot 2021-06-24 at 8 08 23 AM

Option2: Force the tempPath mode to be 755 by changing:

const getMode = (path) => (fs.statSync(path).mode & parseInt('755', 8)).toString(8);

However, this assumes customer's env will generate a tempPath either 755 or 775, which does not guaranteed that this test case won't fail again locally.

Option3: Remove this test case and open a new issue to investigate alternative ways to write this test case

This PR uses option 3. The meaning of this test case is to test whether extractArchive function extract files with same
permission when they archive. However, getMode function in this test suite is system env dependent which doesn’t serve the purpose of this test case.

Signed-off-by: Anan Zhuang [email protected]

Issues Resolved

#4

Test results

unit test for zip.test.js

yarn test:jest /home/anan/work/OpenSearch-Dashboards/src/cli_plugin/install/zip.test.js
yarn run v1.22.5
$ node scripts/jest /home/anan/work/OpenSearch-Dashboards/src/cli_plugin/install/zip.test.js
 PASS  src/cli_plugin/install/zip.test.js
  opensearchDashboards cli
    zip
      ✓ handles a corrupt zip archive (2 ms)
      analyzeArchive
        ✓ returns array of plugins (26 ms)
      extractArchive
        ✓ extracts files using the extractPath filter (10 ms)

Test Suites: 1 passed, 1 total
Tests:       3 passed, 3 total
Snapshots:   3 passed, 3 total
Time:        1.884 s

Overall test result:
Screen Shot 2021-06-24 at 8 41 48 AM

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

@opensearch-ci-bot
Copy link
Collaborator

✅   DCO Check Passed 86aa36b

@opensearch-ci-bot
Copy link
Collaborator

✅   DCO Check Passed 3112862

@tmarkley
Copy link
Contributor

Maybe we should be questioning the actual expectations of the test. If we merely want to ensure that one file is executable and another isn't for everyone, we don't need to match exact file permissions. For example, should we also accept write and execute permissions with 333?

I think we should make the name of the test less vague and refactor so that it's not environment specific.

yuxitang-amzn
yuxitang-amzn previously approved these changes Jun 24, 2021
@ananzh ananzh mentioned this pull request Jun 24, 2021
26 tasks
@ananzh
Copy link
Member Author

ananzh commented Jun 24, 2021

Maybe we should be questioning the actual expectations of the test. If we merely want to ensure that one file is executable and another isn't for everyone, we don't need to match exact file permissions. For example, should we also accept write and execute permissions with 333?

I think we should make the name of the test less vague and refactor so that it's not environment specific.

After investigation and discussion, I suggest to remove this specific test case. getMode function is system env dependent. Remove path.resolve won't work:

expect(getMode(tempPath+'/executable')).toEqual('755');
expect(getMode(tempPath+'/not-executable')).toEqual('644'); 

The above test has same result.

The meaning of this test case is to test whether extractArchive function extract files with same permission when they archive:

expect(getMode('/executable')).toEqual('777');
expect(getMode('/not-executable')).toEqual('644'); 

It should not dependent on system env. Under this test suite, the getMode function can not test this and there is no other easy way to test mode in this test case. I will remove the test case for now and will open an issue.

@opensearch-ci-bot
Copy link
Collaborator

✅   DCO Check Passed dd1bf77

tmarkley
tmarkley previously approved these changes Jun 25, 2021
@ananzh
Copy link
Member Author

ananzh commented Jun 26, 2021

Follow up issue: #561

@tmarkley
Copy link
Contributor

Can you update the title and description since the test is now being removed?

@kavilla kavilla linked an issue Jun 28, 2021 that may be closed by this pull request
1 task
kavilla
kavilla previously approved these changes Jun 28, 2021
Copy link
Member

@kavilla kavilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When squashing please make sure the commit message references that we just removed this test, otherwise if it gets merged with the titled it will be confusing when looking at the history.

With existing fixture, test verify consistency of modes of files 
inside zip.test.js fails. This is due to diff file mode in different
system when doing same command or operation. For example,
in our ubuntu, when create a file using mkdir cmd, the file mode
is 775. But local machine, the file mode is 755.

After unzip executable has mode 777 and not-executable has
mode 644. These two numbers are fixed. They are not dependent
on diff system or folder mode where you unzip the files. However,
the following two results:
path.resolve(tempPath,'executable')
path.resolve(tempPath,’not-executable')
are dependent on tempPath mode because it will use the lower mode.
For example (use mode# directly below):
1)If tempPath is 775 then:
path.resolve(775, 777) —> 775
path.resolve(775, 644) —> 644
2)If tempPath is 755 then:
path.resolve(755, 777) —> 755
path.resolve(755, 644) —> 644
Therefore, this unit test case result is dependent on the mode of
tempPath which is affected by your system or system settings.

After investigation and discussion, this PR remove this
specific test case. The meaning of this test case is to test
whether extractArchive function extract files with same
permission when they archive. However, getMode function
in this test suite is system env dependent which doesn’t
serve the purpose of this test case.

Since it is a broken test, we will remove this one and open
an issue to investigate alternative ways to check file mode.

Issues resolved:
opensearch-project#4

Signed-off-by: Anan Zhuang <[email protected]>
@opensearch-ci-bot
Copy link
Collaborator

✅   DCO Check Passed 304372d

@kavilla kavilla self-requested a review June 28, 2021 20:28
@ananzh ananzh changed the title [Test] Enable unit test case in zip.test.js [Test] Remove unit test case in zip.test.js Jun 29, 2021
@ananzh
Copy link
Member Author

ananzh commented Jun 29, 2021

Can you update the title and description since the test is now being removed?

Done. Thanks for reminding me.

Copy link
Contributor

@tmarkley tmarkley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for updating the details!

@ananzh ananzh merged commit 8b63234 into opensearch-project:main Jun 29, 2021
kavilla pushed a commit that referenced this pull request Jun 30, 2021
With existing fixture, test verify consistency of modes of files 
inside zip.test.js fails. This is due to diff file mode in different
system when doing same command or operation. For example,
in our ubuntu, when create a file using mkdir cmd, the file mode
is 775. But local machine, the file mode is 755.

After unzip executable has mode 777 and not-executable has
mode 644. These two numbers are fixed. They are not dependent
on diff system or folder mode where you unzip the files. However,
the following two results:
path.resolve(tempPath,'executable')
path.resolve(tempPath,’not-executable')
are dependent on tempPath mode because it will use the lower mode.
For example (use mode# directly below):
1)If tempPath is 775 then:
path.resolve(775, 777) —> 775
path.resolve(775, 644) —> 644
2)If tempPath is 755 then:
path.resolve(755, 777) —> 755
path.resolve(755, 644) —> 644
Therefore, this unit test case result is dependent on the mode of
tempPath which is affected by your system or system settings.

After investigation and discussion, this PR remove this
specific test case. The meaning of this test case is to test
whether extractArchive function extract files with same
permission when they archive. However, getMode function
in this test suite is system env dependent which doesn’t
serve the purpose of this test case.

Since it is a broken test, we will remove this one and open
an issue to investigate alternative ways to check file mode.

Issues resolved:
#4

Signed-off-by: Anan Zhuang <[email protected]>
kavilla pushed a commit that referenced this pull request Jun 30, 2021
With existing fixture, test verify consistency of modes of files 
inside zip.test.js fails. This is due to diff file mode in different
system when doing same command or operation. For example,
in our ubuntu, when create a file using mkdir cmd, the file mode
is 775. But local machine, the file mode is 755.

After unzip executable has mode 777 and not-executable has
mode 644. These two numbers are fixed. They are not dependent
on diff system or folder mode where you unzip the files. However,
the following two results:
path.resolve(tempPath,'executable')
path.resolve(tempPath,’not-executable')
are dependent on tempPath mode because it will use the lower mode.
For example (use mode# directly below):
1)If tempPath is 775 then:
path.resolve(775, 777) —> 775
path.resolve(775, 644) —> 644
2)If tempPath is 755 then:
path.resolve(755, 777) —> 755
path.resolve(755, 644) —> 644
Therefore, this unit test case result is dependent on the mode of
tempPath which is affected by your system or system settings.

After investigation and discussion, this PR remove this
specific test case. The meaning of this test case is to test
whether extractArchive function extract files with same
permission when they archive. However, getMode function
in this test suite is system env dependent which doesn’t
serve the purpose of this test case.

Since it is a broken test, we will remove this one and open
an issue to investigate alternative ways to check file mode.

Issues resolved:
#4

Signed-off-by: Anan Zhuang <[email protected]>
@ananzh ananzh deleted the i-4 branch September 14, 2021 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[PURIFY] Fix broken / transient unit test cases
5 participants